Jenkinsfile idiosynchrasies with escaping and quotes

Jenkinsfile.groovy

node {
    echo 'Results included as an inline comment exactly how they are returned as of Jenkins 2.121, with $BUILD_NUMBER = 1'
    echo 'No quotes, pipeline command in single quotes'
    sh 'echo $BUILD_NUMBER' // 1
    echo 'Double quotes are silently dropped'
    sh 'echo "$BUILD_NUMBER"' // 1
    echo 'Even escaped with a single backslash they are dropped'
    sh 'echo \"$BUILD_NUMBER\"' // 1
    echo 'Using two backslashes, the quotes are preserved'
    sh 'echo \\"$BUILD_NUMBER\\"' // "1"
    echo 'Using three backslashes still results in only preserving the quotes'
    sh 'echo \\\"$BUILD_NUMBER\\\"' // "1"
    echo 'To end up with \" use \\\\\\" (yes, six backslashes)'
    sh 'echo \\\\\\"$BUILD_NUMBER\\\\\\"'
    echo 'This is fine and all, but we cannot substitute Jenkins variables in single quote strings'
    def foo = 'bar'
    sh 'echo "${foo}"' // (returns nothing)
    echo 'This does not interpolate the string but instead tries to look up "foo" on the command line, so use double quotes'
    sh "echo \"${foo}\"" // bar
    echo 'Great, more escaping is needed now. How about just concatenate the strings? Well that gets kind of ugly'
    sh 'echo \\\\\\"' + foo + '\\\\\\"' // \"bar\"
    echo 'We still needed all of that escaping and mixing concatenation is hideous!'
    echo 'There must be a better way, enter dollar slashy strings (actual term)'
    def command = $/echo \\\"${foo}\\\"/$
    sh command // \"bar\"
    echo 'String interpolation works out of the box as well as environment variables, escaped with double dollars'
    def vash = $/echo \\\"$$BUILD_NUMBER\\\" ${foo}/$
    sh vash // \"3\" bar
    echo 'It still requires escaping the escape but that is just bash being bash at that point'
    echo 'Slashy strings are the closest to raw shell input with Jenkins, although the non dollar variant seems to give an error but the dollar slash works fine'
}

for build number you could also just do this i believe? ${env.BUILD_NUMBER}

I think "idiosynchrasies" is being overly polite!

thanks, you saved my day

@nowayride I was struggling with interpolation in multiline strings this morning...

FWIW, my issue was that interpolation is NOT supported in single and triple single quoted strings: http://docs.groovy-lang.org/latest/html/documentation/#_string_interpolation

so interpolating in ''' did not work, whereas interpolating in """ worked just fine.

Hi I'm having some issues trying to do this :
sh """ sed -i -e 's/%1//g' \${variable}/foo.txt """
Looks like the "%" is getting some issues in the jenkins file, any idea? also I tried with $1 but it is not working

Thank you for compiling this. Have been fighting some Jenkinsfile string handling oddities, and this helped me nail them down. dollar slashy strings are turning out to be my bestest friend.

Man.. what a mess this is! Eventually when you get it working, its an eye sore in the code.

Thank you for putting this list together.. spent most of the day trying to figure out a simple sh command with a sed replacement ..

I believe Jenkins needs an args: parameter for the "sh" pipeline command, right now we have script:, returnStdout: and returnStatus: both useful to avoid messing with the filesystem. However, is clearly impossible or at least difficult to have clean (and secure) executions, with "sh", just my 2c, maybe Jenkins should do this automatically, but I'm not sure if I want that kind of magic, I rather depend on args.

Like:

sh(script: "echo", args: ["hello", "world", env.MY_ENV, my_other_def])

or a new pipeline command to avoid overloading "sh"

sh(script: "echo", args: ["hello", "world", env.MY_ENV, my_other_def])

On both cases echo the program, not the shell built-in should be executed, Jenkins should look for the program on the systems $PATH/%PATH% but also an absolute path should be supported, like a regular groovy execute on a List.

["/bin/echo", "foo", "bar"].execute()

Is not common to have groovy's execute() allowed on Jenkins sandboxed environment, maybe for good reasons...

I've converted the last comment into a Jenkins issue, let's see what happens. https://issues.jenkins-ci.org/browse/JENKINS-44231

Thank you for making this! To help others in the future, here's how you would do a pipe into sed. Don't need to add extra escapes:

    def imageTag = 'feature/someBranchName'
    def command = $/echo ${imageTag} | sed 's/\//\-/'/$
    
    imageTag = sh(returnStdout: true, script: command).trim()
    sh("echo ${imageTag}")
    // outputs feature-someBranchName

I've found if you use the pipeline snippet generator it will escape your commands for you. Just select "sh: Shell Script" as your Sample Step and enter your command into the text area and click Generate Pipeline Script.

Thank you for creating this, the \\\\\\\ saved me here!

Thank you! This just helped us with a very tricky problem. This is what happens when you let Java programmers write something important.

I was about to burst in tears on my keyboard, then I found this. Thank you! (The fuck Jenkins, why would you do such things?)

'$/curl' -i ........ /$'
/tmp/jenkins8907199902083656367.sh: line 21: $/curl: No such file or directory

doesn't work?

@jurgenweber not sure but is your line actually

'$/curl' -i ........ /$'

With the quotes included? Looks like you have an extra quote in there, but even then the $/ replaces quotes so try

$/curl -i http://example.com//$

Not sure if you need to escape the forward slashes on the URL but likely will need http://example.com//$ if so.

The time I waste on quoting in Jenkinsfiles is rediculous. Thank you for putting this together.

Even with this guide I just spent 3+ hours with this :@

Seriously, groovy escaping in pipeline needs to be changed. It gets to a point that it is stupid and ridiculous to use. I used to love PIPELINE, but replacing bash scripts with groovy is one of the worst thing I ever done in my 20 years of programming.

We have oneliners in bash that is just impossible to escape in groovy. So we are now forced to execute bash scripts in groovy. Which misses the whole point of using groovy the first place.

Thanks for putting this together. I have a few suggestions. Line 12 needs an edit: "yes, seven backticks" should be "yes, seven backslashes".
Also, I'm not sure what you are trying to say in line 2: 'No quotes in single backticks'. For one, the character (') is a single quote not a single backtick. Secondly, I don't know what you mean by the "No quotes" part of that sentence. Do you mean you don't need to use quotes within an echo within an sh?
When writing something like this, where the subject is already confusing, I think it's very important to use the correct names for the characters so you don't confuse people more:
' single quote
" double quote
` backtick
\ backslash

I'm not finding this to be accurate at all. I'm using a multibranch pipeline with Jenkins 2.116 and all the latest plugins as of April 13th 2018, and the dollar slashy solution still leads to errors.

Easiest way is to use the Pipeline Snippet-Generator, it takes care of all backspacing etc.

@laapsaap Lol!, Indeed i had one..

@tbye-pfp thanks, I was braindumping originally and that was incorrect. This was so long ago I don't remember what I intended since I don't use any backticks in the entire script.

@aaronsilverman do you have an example snipped? This was a while back so they may have changed, but I would hope Jenkins wouldn't be that backwards incompatible. If that's the case I can update too.

@abakhru yeah snippet generator helps a lot but there's sometimes pretty complex cases, and if it is automatically generating a ton of backslashes to do nested escapes it may end up with a lot less maintainable code than using dollar slashy syntax. Depends what you're doing but generators (not just in Jenkins) are typically just a good start and still benefit from human review.

how do i use backslash in pipeline shell script? jenkins keeps trying to put single quotes around my backslash!

something like

sh "find . -name *.xml -exec ls \\;"

is becoming...

find . -name '*.xml' -exec ls '\';

Thanks for making the edits! I think it's more clear, and now I understand what you are saying in lines 2 and 4.

Maybe would be useful

bash script

$ cat test.sh
#!/bin/bash

args='-Dexec.args='"'"'${project.groupId}'"'"''
echo "${args}"

$ ./test.sh
-Dexec.args='${project.groupId}'

Pipeline script

node('master'){
   stage('STAGE1'){
      cmd_args='-Dexec.args=\\\'\\${project.groupId}\\\''
      sh "echo $cmd_args"
   }

[Pipeline] node
Running on Jenkins in /var/jenkins_home/workspace/bash-var-expansion
[Pipeline] {
[Pipeline] stage
[Pipeline] { (STAGE1)
[Pipeline] sh
[bash-var-expansion] Running shell script
+ echo -Dexec.args='${project.groupId}'
-Dexec.args='${project.groupId}'
[Pipeline] }

For anyone that ends up here, the Snippet Generator is your best friend.

@Faheetah Nice gist ! Could you add %, that you have to double ? Like sh "echo '1 % 2'" doesn't display the %, you have to double it : "1 %% 2"

seeing this gist made me realized I am not alone on this pain "yay"

I am stuck with this for two friggin days :-/ .

sh 'docker exec tnd-chrome bash -c sudo mvn -q -f tnd-test-automation/pom.xml --settings tnd-test-automation/maven-bdd-settings.xml -Dbrowser=chrome -Dweb-driver=chromedriverlinux.bin -Dlog-level=INFO -Dssh.username=${sshUsername} -Dssh.password=${sshPassword} -Denv=${tndEnv} -Dcucumber.options=\'--tags @Regression\' -B -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=warn clean verify'

this causes sudo to throw usage usage: sudo -h | -K | -k | -V etc...

		sh "docker exec tnd-chrome bash -c \\'sudo mvn -q -f tnd-test-automation/pom.xml --settings tnd-test-automation/maven-bdd-settings.xml -Dbrowser=chrome -Dweb-driver=chromedriverlinux.bin -Dlog-level=INFO -Dssh.username=${sshUsername} -Dssh.password=${sshPassword} -Denv=${tndEnv} -Dcucumber.options=\\\'--tags @Regression\\\' -B -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=warn clean verify\\'"

this causes mvn: -c: line 0: unexpected EOF while looking for matching `''
mvn: -c: line 1: syntax error: unexpected end of file

:/ Any advice greatly appreciated

		sh "docker exec tnd-chrome bash -c 'sudo mvn -q -f tnd-test-automation/pom.xml --settings tnd-test-automation/maven-bdd-settings.xml -Dbrowser=chrome -Dweb-driver=chromedriverlinux.bin -Dlog-level=INFO -Dssh.username=${sshUsername} -Dssh.password=${sshPassword} -Denv=${tndEnv} -Dcucumber.options=\\\'--tags @Regression\\\' -B -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=warn clean verify'"

unexpected EOF while looking for matching `''

Tears of joy :') :')

This worked!!!

sh "docker exec tnd-chrome bash -c 'sudo mvn -q -f tnd-test-automation/pom.xml --settings tnd-test-automation/maven-bdd-settings.xml -Dbrowser=chrome -Dweb-driver=chromedriverlinux.bin -Dlog-level=INFO -Dssh.username=${sshUsername} -Dssh.password=${sshPassword} -Denv=${tndEnv} -Dcucumber.options=\"--tags @Regression\" -B -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=warn clean verify'"

The situation is much worse than this. Any space-separated arguments inside of quotes sent to Jenkins' awful sh task are broken up, single-quoted, and recombined against the mangled outer quoting. Christ, just pass the data through as it was input!

All of this works as expected, people.

The quotes are not dropped by anything in Jenkins. They are interpreted by your shell. Try running some of those commands in bash manually, and you'll see the same behaviour. That is,
echo a
and
echo "a"
will both output the single character a. When you see the shell command being run in the log (the lines that begin with a +), it is not Jenkins showing it to you before sending it to the shell; rather, it is the shell itself echoing it back, as started with the -x (aka -o xtrace) flag.

The backslashes on the other hand are being processed by both Jenkins and the shell. That is, when you write \\\\\\\" as you show above, Jenkins eats one set of escapes, and sends \\\" to the shell; then the shell does one more round, sending a literal \" to echo.

Note, however, that you almost never want this, most executables do not expect literal quotes in their parameters – those are for the shell.

You can easily test that things are working as intended with a shell script that lists its parameters line-by-line:

#!/bin/bash
echo "Got $# args:"
while [ $# -gt 0 ]; do
    echo "$1"
    shift
done
echo

Save this as something like printargs.sh and then replace every echo in your example with it.

Oh, and replace $BUILD_NUMBER with some variable that has spaces in its value, eg:

node {
    withEnv(['FOO=a b c']) {
        sh 'printargs.sh $FOO'
        sh 'printargs.sh "$FOO"'
    }
}

Quoth the @rudolfwalter:

All of this works as expected, people.

The quotes are not dropped by anything in Jenkins. They are interpreted by your shell.

Exactly right. Taking it line-by-line, starting at https://gist.github.com/Faheetah/e11bd0315c34ed32e681616e41279ef4#file-jenkinsfile-groovy-L4-L5 :

   echo 'Double quotes are silently dropped'
   sh 'echo "$BUILD_NUMBER"'

I'll use $USER as my example variable, since it's (pretty much) always present in Bash, whereas $BUILD_NUMBER only exists inside Jenkins runs. So, in Bash, this is equivalent to typing:

echo "$USER"

Which, as expected, generates just bmdan.

Next up, https://gist.github.com/Faheetah/e11bd0315c34ed32e681616e41279ef4#file-jenkinsfile-groovy-L6-L9 :

    echo 'Even escaped with a single backslash they are dropped'
    sh 'echo \"$BUILD_NUMBER\"'
    echo 'Using two backslashes, the quotes are preserved'
    sh 'echo \\"$BUILD_NUMBER\\"'

What we're seeing here is Groovy's escaping. The rules are as follows:

1. After sh, look for the next string literal opener.
2. Load the string (honoring escapes), until you hit the corresponding (unescaped) closer for your opener.
3. Send the string to the system(3) or execve(2) call.

In the first case, Groovy helpfully honors a meaningless escape (\" inside of single quotes is just a literal ", same as it would have been without the \). Thus, after step 2, we're sending echo "$BUILD_NUMBER" to the shell, same as the previous two lines. In the second case, Groovy's escaping drops the \\ down to just a \, and sends that through, so the final command sent to the shell is echo \"$BUILD_NUMBER\". Try that as echo \"$USER\" in your shell and you'll see the same output: quotes around the var.

Moving on to https://gist.github.com/Faheetah/e11bd0315c34ed32e681616e41279ef4#file-jenkinsfile-groovy-L10-L11 :

    echo 'Using three backslashes still results in preserving the single quotes'
    sh 'echo \\\"$BUILD_NUMBER\\\"'

Three backslashes followed by a " now makes perfect sense, because Groovy sees this as two escapes: the first, an escaped backslash. The second, a (needlessly-) escaped ". So when this goes into the shell, it's echo \"$BUILD_NUMBER\", indistinguishable from the two slashes example above it.

Finally, we have https://gist.github.com/Faheetah/e11bd0315c34ed32e681616e41279ef4#file-jenkinsfile-groovy-L12-L13 :

    echo 'To end up with \" use \\\\\\\" (yes, seven backslashes)'
    sh 'echo \\\\\\"$BUILD_NUMBER\\\\\\"'

First off, no, you have six backslashes here. Which is exactly how many it should take. The final string sent to the shell is echo \\\"$BUILD_NUMBER\\\": an escaped \ and an escaped ", concatenated, on each side of the variable. It takes six backslashes to make sure that three backslashes end up at the shell once Groovy's done with it. Mind you, doing this is almost certainly not what you wanted to do; this prints out literal quotes and backslashes. If you're doing that in preparation for loading this into something else, you want to escape the contents of the variable, too, or something like echo \\\\\\"$PATH\\\\\\" is going to go straight to hell if there's an embedded " or \ inside $PATH.

Bonus: if you do happen to use seven backslashes, though, you get the exact same output. Think through why: any odd number of backslashes is simply going to hand some number of backslashes to the shell, and then escape the character following the last backslash in Groovy. Thus, final backslash will never show up by the time the shell gets it, the same as the 1- and 3-backslash formulations, above.

@michaelPf85 added, thanks!

I'm surprised this gist is still so active, good info everyone. I don't always see replies so best bet for more prompt feedback would probably be to post on SO or something, but I'll try to keep this updated with suggestions as I see them.

Edit: um, this doesn't look to be the case. When I use sh 'echo something with %%' it comes out with both of them.

@CH-DanReif I'm going to test run this and include the results as of Jenkins 2.121 since it should help a bit, and let me error check this (might have been updates since I wrote it). Looks like it is indeed six backslashes and I've updated the echo to note that, although seven technically works because of the aforementioned triple backslash. And yes there are some better ways of handling that but there are use cases for passing a literal "$var" to the shell. This gist doesn't reflect best bash practices but rather how to get around how clunky backslashes are going through several layers of escaping. It may not be a var but a string with a dollar sign in it too.

So how would I escape this dos batch command for a Jenkinsfile bat command?

%OC_EXE% -register:user -target:"C:\Program Files\dotnet\dotnet.exe" -targetargs:"test DNC.XTest\DNC.XTest.csproj --test-adapter-path:. --logger:xunit;LogFilePath=..\xunit-test-results.xml" -output:"%RAWOUTPUT%" -filter:+[*]* -oldStyle

slashy string don't seem to work, or I haven't nutted it yet.

In my Jenkinsfile I have this:
bat "${env.OC_EXE} -register:user -target:${env.DOTNET_EXE} -targetargs:'test --test-adapter-path:. --logger:xunit;LogFilePath=../${env.UT_RESULTS}' -output:../${env.RAWOUTPUT} -filter:+[*]* -oldStyle"

Which I can see in the console produces this:
G:\Jenkins\workspace\POC\Pipeline\DNCTest\DNC.XTest>C:\Users\xxx/.nuget/packages/opencover/4.6.519/tools/OpenCover.Console.exe -register:user -target:C:/PROGRA~1/dotnet/dotnet.exe -targetargs:'test --test-adapter-path:. --logger:xunit;LogFilePath=../xunit-test-results.xml' -output:../raw_coverage.xml -filter:+[*]* -oldStyle

But is throwing this error:
Incorrect Arguments: The argument '--test-adapter-path' is not recognised

How the heck do I correctly wrap -targetargs correctly so groovy will accept it and OpenCover knows to just pass --test-adapter-path onto dotnet.exe??

@agray this comes more down to how does Jenkins handle and escape spaces. This is a blight on any code trying to wrap shell commands, you come across it in any language and framework. Usually it's done with lists separating args so you get something like:

doShell(['ls', '-ahl', 'some directory'])

Which gets interpreted the same as running:

$ ls -ahl some\ directory

Jenkins, from all I've seen, has decided to use only strings as shell input. If you are dealing with too many types of things to escape, it might be easier to use an explicit script then call that. I haven't used Jenkins with Windows scripting but with Linux I believe you have to escape each quote or it disappears, so you would have to do something akin to:

sh 'foo --bar=\\\"something with spaces\\\"'

In my case the default shell happened to be 'dash', so a saner solution was to just switch to bash:

sh "#!/bin/bash\n" + """echo "with 'quotes'!" """

dollar-slashy should work too, and no excaping needed