Friday, November 17, 2017

Fixing 'VOC' testing on Windows

Over the Veteran's Day weekend, I was looking at the issue tracker for VOC and noticed several similar issues relating to testing on Windows. So I brought out a Windows (XP) machine, and set to work installing dependencies. Because it was XP, some of the components were not the most up-to-date.
  • Python 3.4.4 is the latest version build for XP.
  • Java 1.8 (u152) works, but warns loudly that it is not really supported on XP.
  • Apache Ant 1.10.1 is the latest, and seems to work fine.
  • Git 2.10.0 is this is the latest available for XP.
After getting these all configured, and the environment variables set up...
  1. The shlex module from ouroboros was failing to build. This was because of UTF-8 characters that were not being accounted for in voc\transpiler.py. I have submitted a PR for this, and it has been merged.
  2. I ran into an issue that appears to be a problem with the flush() in runAsJava. Once I dug down far enough to find the actual error, I found a StackOverflow question that pointed me in the right direction. Popen(['java'], ...) was picking up java.exe from C:\Windows\System32 instead of the one in the %PATH%, so I had to rename the former.
  3. Due to performing this work on an under-powered machine, I had to increase the timeout for calling 'ant java' from tests\utils.py from 30 seconds to a minute.
After this, I finally got to the issue that was being experienced by so many others. For test_str tests, Python was emitting a UnicodeEncodeError because it was trying to use the encoding 'cp437' and some of the characters in the test do not map. The Java version was emitting UTF-8 characters, and while the console doesn't understand that encoding and displays them incorrectly, it does not throw an exception. It is this difference which causes the tests to fail.

While Python 3.6 apparently gets UTF-8 working correctly, a work-around for this issue on earlier versions of Python is done by setting the environment variable PYTHONIOENCODING to 'UTF-8'. I have had a fix ready for several days but have only had a little time here and there to test it. I have submitted a PR and will just see where it goes.

Friday, October 13, 2017

Bitten by cdrkit bug

At work, I recently ran into a problem that I eventually tracked down to a bug in cdrkit.  This glosses over a few steps, but should capture the essentials.

A computer that, among other things, is used for burning DVD's recently made the move from RHEL (Red Hat Enterprise Linux) version 5 to version 7.  We realized shortly there-after that it was failing to burn DVD's successfully.  After some testing, we found that the same problem occurred on another box with the exact same model of DVD drive.

After confirming that the model of DVD drive would still write using RHEL5, I took the cdrecord binary from RHEL5, ran it under RHEL7, and it worked.  Clearly the difference was between cdrecord (from cdrtools, used on RHEL5) and wodim (from cdrkit, which is aliased to cdrecord on RHEL7).

I ran both through strace, and found that for fixating (finalizing) the disc, cdrecord was using a timeout of 1000 seconds, while wodim was using 200 seconds.  On the drive in question, fixating the drive takes about 240 seconds.  So with wodim, the timeout is hit, some diagnostic codes are dumped, and wodim reports that the fixate process failed.

Digging into source code, I found that one of Red Hat's patches for cdrtools was injecting a function (fixate_mdvd) that extends the timeout to 1000 seconds, calls the original cdrecord fixate function (fixate_mmc), and restores the 200-second timeout.

That Red Hat patch became part of the cdrkit fork, but additional changes 1 made in 1.1.6 interfere with it.  In cdrtools, the time-consuming part of the operation is happening inside of scsi_flush_cache, which is called by fixate_mmc.  The change in 1.1.6 makes an additional call to scsi_flush_cache from fixate_mdvd, before it extends the timeout.  This essentially negates the effect of the timeout change for Disk-At-Once sessions.  This 1.1.6 change appears to have been made to solve a Debian bug 2, but I don't see the evidence that it helped anything. Even if it did appear to help, it may have just been hiding the real cause.

I believe the 1.1.6 change to fixate_mdvd should be reverted to fix the present bug.  If the bug in the Debian report is encountered again, more rigor needs to go into determining the root cause.


1 It's hard to find code history for cdrkit, but the change can be seen here: http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/gutsy/cdrkit/gutsy/revision/8/wodim/drv_mmc.c
2 Debian Bug #411362: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=411362

Thursday, October 12, 2017

Book Summary: Do Over by Jon Acuff


While I'm only commuting 4-5 hours in total most weeks, I want to make better use of that time, and I am getting a bit tired of just music on the radio.  Last week, I decided to start with listening to Do Over by Jon Acuff, and because I'm no quitter, I just reached the finish line with it.  As indicated by the title, this is not so much a review, as just a summary to help me remember some key points.


The core of the book is about making investments into your CSA.  That would be your Career Savings Account.  The components of the CSA are skills, relationships, character, and hustle.  Any time you encounter a career transition, voluntary or not, a healthy CSA will be a valuable asset.

That old phrase "it's not what you know, it's who you know" speaks to the importance of relationships to your career.  There is a lot of truth to this.  You may think you can pull yourself up by your own bootstraps, but the world was not designed that way.  We all need each other.


The skills that you know are still important though.  Relationships may help you get your foot in the door, but without skills, you'll still have a hard time getting to the next one.  In addition to the technical skills we may think of, soft skills like showing up reliably, diplomacy, and flexibility are valuable.

Character is who you are.  It will bleed into everything you do and touch everyone you know.  Jon likens it to an orchard, as it takes time and constant tending to develop it.  You won't see the fruit for a while, but it will come.

Hustle will have an amplifying effect on the other investments.  Frequency is key to strong relationships.  Time and effort are essential to sharpen and build new skills.  Constancy is required to grow character.  Hustle is about putting in the time, and time is key.

With a healthy, diversified CSA, you will be ready for any career transition, whether that be breaking through a ceiling, making a jump, recovering from a bump, or taking advantage of an opportunity.

Monday, June 2, 2014

Google Play Books 3.1.49

I saw a Google Play Books update was available and decided to do a quick teardown.  The most interesting thing I saw was a new Offers activity.  I'm not sure exactly where it will show up, but here is roughly what it will look like:

Friday, November 22, 2013

Every. Single. Time.

The Play Store app is somewhat special among Android apps, in that it is updated silently.  Google recently updated their Play Store app to use the new drawer layout.  It has been rolling out over a several-week span.

When I got it a couple of weeks ago, I realized it only because the menu went from around six items, down to just two.  Fortunately, I had read that this update was coming, so I figured out pretty quickly that the other menu items had been moved to a drawer and all was well.

A few days later, my son couldn't find the 'installed apps' section.  I showed him the new drawer and all was well.

More recently, my elder daughter couldn't find the 'my wishlist' section.  I showed her the new drawer and all was well.

I know other cases of people encountering this little bump in the road.  On Google's Android development site, there is discussion about the relatively-new drawer pattern.  There is even discussion of how to introduce users to your app's drawer, so they won't be confused.  Too bad the Google Play Store app developers couldn't have paid a little closer attention to Google's developer suggestions.

Sunday, November 17, 2013

I'll Get Started, After My MTE...

C'mon Jon, I'm just starting chapter three and you're really pulling the rug out from under one of my stand-by excuses.  What do you mean I can start living with purpose today, without having 'found my purpose'?  See here's the deal as I've figured it to this point...

First you're supposed to plan out an extended weekend where you can get away by yourself.  Preferably it should be some rustic cabin in the middle of nowhere, so there are no distractions and you can be alone with God and His creation.  While you're there you spend a ton of time resting and praying.  You have a Mountain-Top Experience (MTE1) with Him, and somewhere in there he gives you a vision revealing your One True Purpose (OTP2) in this life.  Then you can start living life to its fullest.

I'm running out of excuses here Mr. Acuff.  Give me a break, will you?


1. MTE - I don't plan to actually use that acronym again, but the term is thrown around so much in Christian circles that it seemed to deserve an acronym.
2. OTP - Alright, you caught me. I just like acronyms.

Tuesday, November 12, 2013

Improving Android App Tear-downs

Every now and then, I've done app tear-downs a la Ron Amadeo (unpublished though).  My primary tools have been apktool and meld.  I noticed that sometimes there was a lot of noise in the smali folder from changing 0x7f...... literals.  I determined that this was happening because of changes in the resource ID mapping.  For instance R.string.xyz might map to 0x7f010001 in one version of an app, but map to 0x7f010003 in the next version.

So a few months ago, I wrote a small Java program that parsed the ID's in res/values/public.xml and inserted comments in the smali files where matches were found.  I could then add rules in meld to filter this noise out.  I wrote the program in Java with the thought of eventually merging it with apktool.  However, it has just set there.

A few days ago I decided to push it forward.  I contacted the apktool developers on IRC and decided that it should really be integrated with baksmali.  By putting it there, the comments could be generated as the smali files are generated, rather than inserted in a post-process.

Making the changes to baksmali was harder than I expected, but I finally got it working.  It is still a bit rough, but this morning I made a pull request via github.  If it makes it in, there will still be some work to do on apktool, but I believe the hardest part of the coding is past.  Hopefully app tear-down will soon be just a tiny bit easier for everyone.