git: Moving partial changes between commits

Now and then I face the fact that I’ve added changes to a commit I’d like to have moved into a different commit. Here is what you do:

What’s there

We have two commits. For illustration purposes I’ve trimmed the log output down:

$ git log --stat
commit 19c698a9ee91a5f03f1c3240fc957e6b328931f5

    WIP: adding tests

 parts/tests/functional/conftest.py       |  4 ++--
 parts/tests/functional/test_frobfrob.py  | 43 ++++++++++
 frobfrob.py                              | 14 +++++++++++++-

commit c7ef6c3014ca9d049dea46fbed44010acf53ae79

    prepare frob frob schemas

 parts/tests/functional/conftest.py           | 31 +++++++++++++
 frobfrob/models.py                           | 32 +++++++++++++

commit 5b30d351f51fda40d37d2f7dc25d2367bd37845a
[...]

Now I want to move the changes made to conftest.py from commit c7ef6c3014ca9d049dea46fbed44010acf53ae79 into commit 19c698a9ee91a5f03f1c3240fc957e6b328931f5 (or HEAD).

Pluck out the commit

In order to pluck out the changes to conftest.py, we’ll reset the file against the previous commit 5b30d351f51fda40d37d2f7dc25d2367bd37845a (you could also use HEAD~3).

$ git reset 5b30d351f51fda40d37d2f7dc25d2367bd37845a parts/tests/functional/conftest.py
Unstaged changes after reset:
M       parts/tests/functional/conftest.py

$ git status -s
MM parts/tests/functional/conftest.py

As you can see, we will have staged changes and unstaged changes. The staged changes remove the additions to the conftest.py file and the unstaged changes add our code to conftest.py

Remove and Add

We now create two commits:

1. Use the staged changes for a new commit which we’ll squash with c7ef6c3014ca9d049dea46fbed44010acf53ae79.
2. Stage the unstaged changes and create another commit which we’ll squash with 19c698a9ee91a5f03f1c3240fc957e6b328931f5 or HEAD.

# 1. commit Message is something like: squash: removes changes to conftest.py
$ git commit

# 2. commit
# stage changes
$ git add -p

# commit, message will be something like: squash: adds changes to conftest.py
$ git commit

# we end up with two additional commits
$ git log --oneline
492ff22 Adds changes to conftest
8485946 removes conftest files
19c698a WIP: adding tests
c7ef6c3 prepare frob frob schemas

Interactive rebase put’s it all together

Now use an interactive rebase to squash the changes with the right commits:

$ git rebase -i HEAD~5

Haskell: From N00b to Beginner

I gave a talk about my experience learning Haskell:

The slides can be found here: http://redhat.slides.com/rjoost/deck-2

(Locally) Testing ansible deployments

I’ve always felt my playbooks undertested. I know about a possible solution of spinning up new OpenStack instances with the ansible nova module, but felt it to be too complex as a good idea to implement. Now I’ve found a quicker way to test your playbooks by using Docker.

In principal, all my test does is:

1. create a docker container
2. create a copy of the current ansible playbook in a temporary directory and mount it as a volume
3. inside the docker container, run the playbook

This is obviously not perfect, since:

• running a playbook locally vs connecting via ssh can be a different beast to test
• can become resource intensive if you want to test different scenarios represented as docker images.

There is possibly more, but for myself in small it is a workable solution so far.

Find the code on github if you’d like to have a look. Improvements welcome!

 

Ansible Variables all of a Sudden Go Missing?

I’ve written a playbook which deploys a working development environment for some of our internal systems. I’ve tested it with various versions of RHEL. Yet when I ran it against a fresh install of Fedora it failed:

fatal: [192.168.1.233] => {'msg': "One or more undefined variables: 'ansible_lsb' is undefined", 'failed': True}

It turned out, that ansible gets it’s facts through different programs on the remote machine. If some of these programs are not available (in this instance it was lsb_release) the variables are not populated resulting in this error.

So check if all variables you access are indeed available with:

$ ansible -m setup <yourhost>

(lxml) XPath matching against nodes with unprintable characters

Sometimes you want to clean up HTML by removing tags with unprintable characters in them (whitespace, non breaking space, etc). Sometimes encoding this back and forth results in weird characters when the HTML is rendered. Anyways, here is the snippet you might find useful:

def clean_empty_tags(node):
    """
    Finds all tags with a whitespace in it. They come out broke and
    we won't need them anyways.
    """
    for empty in node.xpath("//p[.='\xa0']"):
        empty.getparent().remove(empty)

Common docker pitfalls

I’ve ran into a few problems with docker I’d like to document myself and how to solve them.

Overwriting an entrypoint

If you’ve configured a script as an entrypoint which fails, you can run the docker image with a shell in order to fiddle with the script (instead of continously rebuilding the image):

#--entrypoint (provides a new entry point which is the nominated shell)
docker run -i --entrypoint='/bin/bash'  -t f5d4a4d6a8eb

Possible errors you face otherwise are these:

/bin/bash: /bin/bash: cannot execute binary file

Weird errors when building the image

I’ve ran into this a few times. Errors like:

Error in PREIN scriptlet in rpm package libvirt-daemon-0.9.11.4-3.fc17.x86_64
or
useradd: failure while writing changes to /etc/passwd

If you’ve set SELinux to enforcing, you may want to temporarily disable SELinux for just building the image. Don’t disable SELinux permanently.

Old (base) image

Check if your base image has changed (e.g. docker images) and pull it again (docker pull <image>)

Abort a git commit –amend

The situation

You hack on a patch, add files to the index and with a knee jerk reaction do:

git commit --amend

(In fact, I do this in my editor with the vim-fugitive plug-in, but it also happened in the terminal). For the commit message git places you in your text editor. If you quit, your changes are merged with the last commit. Being aware of your trapped situation, what do you do?

The solution

Simply delete the commit message (up to where the comments start with #). The typical git commit-hook will see it as a commit with an empty message and abort the commit and therefore the merge.

KVM: Resizing a VMs hard disk

When it comes to a full hard disk in one of your virtual machines, I found this article utterly useful resizing it:

KVM Linux – Expanding a Guest LVM File System Using Virt-resize
http://blog.oneiroi.co.uk/linux/kvm/virt-resize/RHEL/LVM/kvm-linux-expanding-a-lvm-guest-file-system-using-virt-resize/

git: rework a commited patch

Since I’m working with gerrit and following a few best practices from colleagues. Here is one I found very helpful.

Motivation

A bad commit can happen from time to time, which perhaps addresses only partially a problem. Best practice is to revert the commit, rework it until it fixes the problem. Why? That way you can grab the “right” patch which fixes the specific problem instead of searching for multiple patches and fix of fixes.

How you do it

1. Create a branch with bad commit on top (a0eae90 is the SHA of the bad commit):

   # puts you in detached head state with a0eae90b45f9b94c3f0742df627eefd864c089c8 at the top
   git checkout a0eae90b45f9b94c3f0742df627eefd864c089c8
   # checkout new branch with the name fixup/bug_2312312 in order to work on it.
   git checkout -b fixup/bug_2312312

2. Revert the patch (HEAD~1 == a0eae90):

   git revert a0eae90b45f9b94c3f0742df627eefd864c089c8

3. Cherry pick the “bad” commit:

   git cherry-pick a0eae90b45f9b94c3f0742df627eefd864c089c8

4. Unstage the cherry picked commit to work on the patch:

    git reset HEAD~1

Kudos to Simon Baird and Peter Hutterer for their help on this.

Touché

I just ran into a problem in which I got myself cornered. I’ve edited a configuration file with elevated permission using sudo:

sudo vim /etc/httpd/conf.d/metrics.conf

From time to time when editing, I background the vim session in order to perform some commands in the shell, then continue working in vim. This time, I’ve sent the editor into the background. When trying to foreground it I got this:

~/Metrics(branch:bug_947304*) » fg
[1]  + 11403 continued  sudo vim /etc/httpd/conf.d/metrics.conf
[1]  + 11403 suspended (tty input)  sudo vim /etc/httpd/conf.d/metrics.conf

What? How? When? Any attempt to foreground this process failed. Reason was the elevated permission I’ve started the editor with. Long story short, you’ll need to kill the editor in order to get back into editing. This mailing list thread explains why.