[Pre­vi­ous­ly published at the now defunct MetaBrite Dev Blog.]

A collection of mis­cel­la­neous tips on using Pipelines in Jenkins 2.0.

#6 in a series on Jenkins Pipelines

withCredentials([
    [$class: 'StringBinding', credentialsId: 'GPG_SECRET', variable: 'GPG_SECRET'],
    [$class: 'AmazonWebServicesCredentialsBinding',
     credentialsId: '0defaced-cafe-f00d-badd-0000000ff1ce',
     accessKeyVariable: 'AWS_ACCESS_KEY_ID',
 

[Pre­vi­ous­ly published at the now defunct MetaBrite Dev Blog.]

Jenkins Pipelines are written in a Groovy DSL. This is a good choice but there are surprises.

#5 in a series on Jenkins Pipelines

[Pre­vi­ous­ly published at the now defunct MetaBrite Dev Blog.]

If there isn't a built-in Pipeline step to accomplish something, you'll almost certainly use the sh step.

#4 in a series on Jenkins Pipelines

The sh step runs the Bourne shell—/bin/sh, not Bash aka the Bourne-again shell—with the -x (xtrace) and -e (errexit) options.

The xtrace option means that every step in the sh block is echoed to the Jenkins log, after commands have been expanded by the shell. This is useful but you could echo the contents of passwords or secret keys in­ad­ver­tent­ly. Use set +x in your sh block to control this.

The errexit option means that the …continue.

[Pre­vi­ous­ly published at the now defunct MetaBrite Dev Blog.]

Much of our code is in one large GitHub repository, from which several different ap­pli­ca­tions are built. When changes are pushed to the master branch, we want only the ap­pli­ca­tions in affected di­rec­to­ries to be built. This was not easy to get right with “Pipeline script from SCM” builds.

#3 in a series on Jenkins Pipelines

[Pre­vi­ous­ly published at the now defunct MetaBrite Dev Blog.]

The “slave” ter­mi­nol­o­gy is un­for­tu­nate, but the utility of running a Jenkins build on a node that you've configured at Amazon's EC2 is undeniable.

#2 in a series on Jenkins Pipelines

We needed to install system packages on our build nodes, such as Docker or Postgres. For obvious reasons, Cloud­Bees—our Jenkins hosting provider—­won't let you do that on their systems. You must provide your own build nodes, where you are free to install whatever you like.

We already use Amazon Web Services, so we chose to configure our CloudBees account with EC2 slaves. We had a long and fruitless detour through On-Premise Executors, which I …continue.

[Pre­vi­ous­ly published at the now defunct MetaBrite Dev Blog.]

The MetaBrite dev team migrated most of their builds from At­las­sian's Bamboo Cloud to Jenkins Pipelines in late 2016/early 2017. This is a series of blog posts about that experience.

[Pre­vi­ous­ly published at the now defunct MetaBrite Dev Blog.]

I attended DockerCon 2016 in Seattle over the last two days and I learned a lot. It was a well-run conference with an en­thu­si­as­tic audience.

I'm astounded at the growth of Docker. Three-and-a-quarter years ago, Docker was revealed to the public for the first time, in a five-minute lightning talk at PyCon 2013. In January 2016, Docker Hub had received 1.6 billion image pulls; by this month, that number had jumped to over 4 billion pulls! DockerCon had over 4,000 attendees and nearly 100 exhibitors, who clearly believe there's a multi-billion dollar market for containers. DataDog concurs, in a report on Docker adoption.

Last night, Adam Porad and I were one of five teams pitching our startups at the PuPPy-organized PyCon Startup Row Pitch Night:

Techstars Seattle and PuPPy [Puget Sound Pro­gram­ming Python] presents PyCon Startup Row Pitch Night. The time has come again for you, the members of PuPPy, to select Seattle’s startup rep­re­sen­ta­tive to travel to PyCon in Portland to represent our Python community and startup scene at the annual conference produced by the Python Software Foundation.

We were pitching MetaBrite and our technology that captures receipts, yielding receipt in­for­ma­tion to users and onsumer insights. We use Python ex­ten­sive­ly—we've written 120,000 lines of Python code for web services, web apps, machine learning, image processing, and …continue.

[Pre­vi­ous­ly published at the now defunct MetaBrite Dev Blog.]

I needed to sort a Python dictionary by value today, rather than by key. I found it confusing, so I'll share what I learned.

Assume the following dictionary, where each value is a tuple of (ID, score). How do we sort by score; i.e., the second item in the value tuple? (For the purposes of this discussion, ignore the meaning of the dic­tio­nary's key.)

>>> some_dict = dict(a=(123, 0.7), b=(372, 0.2), e=(456, 0.85), d=(148, 0.23), c=(502, 0.1))
>>> some_dict
{'a': (123, 0.7), 'c': (502, 0.1), 'b': (372, 0.2), 'e': (456, 0.85), 'd': (148, 0.23)}

Python dic­tio­nar­ies are inherently unsorted, unless you …continue.

[Pre­vi­ous­ly published at the now defunct MetaBrite Dev Blog.]

Some time ago, we made an ill-considered decision to use recipe names for image URLs, which simplified image management with our then-rudi­men­ta­ry tools. For example, the recipe named "Twisted Pasta With Browned Butter, Sage, and Walnuts" becomes a URL ending in "Twist­ed%20­Pas­ta%20With­%20Browned%20But­ter%2C%20Sage%2C%20and%20Wal­nuts.jpg".

Life becomes more in­ter­est­ing when you escape the confines of 7-bit ASCII and use Unicode. How should u"Sautéed crème fraîche Provençale" be handled? The only reasonable thing to do is to first convert the Unicode string to UTF-8 and then hex-encode those octets: "Saut%C3%A9ed%20cr%C3%A8me%20fra%C3%AEche%20Proven%C3%A7ale".

That seems reasonable, but it was giving us in­con­sis­tent results when the images were uploaded to an S3 bucket. When …continue.