Archived Posts from this Category
Archived Posts from this Category
Recently I was writing a Python-language tool, and some of my doctests (text fixtures, within module comments) were failing. When I tried to import the StringIO module in my test, I got a quite annoying message, “Got nothing”, and the test didn’t work as I wanted. I asked StackOverflow. User wim there gave me a crucial insight, but didn’t explain the underlying cause of my problem. I read the doctest code, and came up with an explanation that satisfied me. I am posting it here, as an aid to others. The gist of the insight: What looks like a multi-line doctest fixture is in fact a succession of single-line doctest “Examples”, some which return no useful result but which set up state for later Examples. Each single-line Example should each have a >>> prefix, not a … prefix. But, there are Examples that require the … prefix. The difference lies in Python’s definition of an Interactive Statement.
I posted a question much like this to StackOverflow:
This simplified test case demonstrates the error:
# encoding: utf-8
>>> from StringIO import StringIO
if __name__ == “__main__”:
Expected behaviour: test passes.
Observed behaviour: test fails, with output like this:
File "./src/doctest_fail.py", line 7, in __main__.Dummy
from StringIO import StringIO
s = StringIO()
print(”s is created”)
s is created
1 items had failures:
1 of 1 in __main__.Dummy
***Test Failed*** 1 failures.
Why is this doctest failing? What change to I need to make in order to be able to use StringIO-like functionality (a literal string with a file interface) in my doctests?
(I had originally suspected the StringIO module of being part of the problem. My original question title was, “Why is use of StringIO breaking my doctest (Python 2.7)”. When I realised that suspicion was incorrect, I edited the question on StackOverflow.)
StackOverflow expert wim was quick with the crucial insight: “It’s the continuation line syntax (…) that is confusing doctest parser.” Wim then rewrote my example so that it functioned correctly. Excellent! Thank you, wim.
I wasn’t satisfied, however. I know from didn’t explain the underlying cause of my problem. I read the doctest code, and came up with an explanation that satisfied me. Below is an improved version of the answer I posted to StackOverflow at the time.
# encoding: utf-8
>>> from StringIO import StringIO
>>> s = StringIO()
if __name__ == “__main__”:
Now the corrected example, renamed
doctest_pass.py, runs with no errors. It produces no output, meaning that all tests pass:
Why is the
>>> syntax correct? The Python Library Reference for doctest, 18.104.22.168. How are Docstring Examples Recognized? should be the place to find the answer, but it isn’t terribly clear about this syntax.
Doctest scans through a docstring, looking for “Examples”. Where it sees the PS1 string
>>>, it takes everything from there to the end of the line as an Example. It also appends any following lines which begin with the PS2 string
... to the Example (See:
_EXAMPLE_RE in class
doctest.DocTestParser, lines 584-595). It takes the subsequent lines, until the next blank line or line starting with the PS1 string, as the Wanted Output.
An “interactive statement” is a statement list ending with a newline, or a Compound Statement. A compound statement, e.g. an
try statement, “in general, […spans] multiple lines, although in simple incarnations a whole compound statement may be contained in one line.” Here is a multi-line compound statement:
if 1 > 0:
A statement list is one or more simple statements on a single line, separated by semicolons.
from StringIO import StringIO
s = StringIO(); print("s is created")
So, the question’s doctest failed because it contained one Example with three simple statements, and no semicolon separators. Changing the PS2 strings to PS1 strings succeeds, because it turns the docstring into a sequence of three Examples, each with one simple statement. Although these three lines work together to set up one test of one piece of functionality, they are not a single test fixture. They are three tests, two of which set up state but do not really test the main functionality.
By the way, you can see the number of Examples which
doctest recognises by using the
-v flag. Note that it says, “
3 tests in __main__.Dummy“. One might think of the three lines as one test unit, but
doctest sees three Examples. The first two Examples have no expected output. When the Example executes and generates no output, that counts as a “pass”.
% ./src/doctest_pass.py -v
1 items had no tests:
1 items passed all tests:
3 tests in 2 items.
3 passed and 0 failed.
Within a single docstring, the Examples are executed in sequence. State changes from each Example are preserved for the following Examples in the same docstring. Thus the
import statement defines a module name, the
s = assignment statement uses that module name and defines a variable name, and so on. The doctest documentation, 22.214.171.124. What’s the Execution Context?, obliquely discloses this when it says, “examples can freely use … names defined earlier in the docstring being run.”
The preceding sentence in that section, “each time doctest finds a docstring to test, it uses a shallow copy of M’s globals, so that … one test in M can’t leave behind crumbs that accidentally allow another test to work”, is a bit misleading. It is true that one test in M can’t affect a test in a different docstring. However, within a single docstring, an earlier test will certainly leave behind crumbs, which might well affect later tests.
But there is an example doctest, in the Python Library Reference for doctest, 126.96.36.199. How are Docstring Examples Recognized?, which uses
... syntax. Why doesn’t it use
>>> syntax? Because that example consists of an
if statement, which is a compound statement on multiple lines. As such, its second and subsequent lines are marked with the PS2 strings. It’s unfortunate that this is the only example of a multi-line fixture in the documentation, because it can be misleading about when to use PS1 instead of PS2 strings.
I use the GnuCash free software accounting software. Like many accounting tools, it can import bank or credit card transactions, and has a way to learn the correct mapping from transaction data to my own account structure. And, sometimes the tool gets the mapping wrong, and needs to be reset. Here is how I was able to perform this reset. I post it here in the hopes it will help others. Fasten your seatbelt, it involves some awfully technical command-line tools, including XSLT processing. Continue Reading »
I just resolved a problem installing a Python module pycdio on my Mac OS X 10.10.1 “Yosemite” operating system. The error message was obscure: “Error: Unable to find ‘python.swg’”, and “Error: Unable to find ‘typemaps.i’”. The solution involved something non-obvious about how Mac Ports handles swig. Here’s my notes, in hopes of helping others seeing this error.
I use and value a good spreadsheet application the way chefs use and value good knives. I have countless occasions to do ad-hoc data processing and conversion, and I tend to turn to spreadsheets even more often I turn to a good text editor. I know a lot of ways to get the job done with spreadsheets. But recently I learned a new trick. I’m delighted to share it with you here.
The situation: you have an HTML document, with a list of linked text. Imagine a list of projects, each with a link to a project URL (the names aren’t meaningful):
The task is to convert this list of formatted links into a table, with the project name in column A, and the URL in column B. The trick is to use an OpenOffice macro, which exposes the URL (and other facets of formatted text) as OpenOffice functions. Continue Reading »
I post on various forums around the net, and a few of my posts there get some very gratifying kudos. I’ve been a diligent contributor to StackOverflow, the Q-and-A site for software developers. I’m in the top 15% of contributors overall, and one of the top 25 answerers of Unicode-related questions. Here’s my second best-voted answer in StackOverflow so far.
The question, How do I get SQLAlchemy to correctly insert a unicode ellipsis into a mySQL table?, was asked by user kvedananda in February 2012. In abbreviated form, it was:
Canada Post and the US Postal Service raised their postage rates again in January 2013. I was busy then, but I’ve grabbed a moment and updated my handy Canada Post and USPS postage rate quick reference card. The Canada Post rate increases were effective January 14, 2013, and the USPS increases were effective January 27.
My Canada Post and USPS Postage Rates project page, http://jdlh.com/en/pr/postage_card.html, has links to download the latest charts as I update them. The spreadsheet source file for the charts is also there. Both are licensed CC-BY-SA, so please feel free to re-use and modify them (as long as you attribute my work and share your product as freely).
Heads up: Canada Post has already received approval for first-class mail rate increases in 2014. The 2013 increases of both agencies came almost exactly one year after their 2012 increases, so I won’t be surprised if this becomes an annual event. The good news is that both Canada Post and USPS offer “perpetual” or “forever” stamps, which are worth first-class basic domestic postage, whatever the price may increase to.
With the year coming to an end, it is the season of making donations to organisations doing good in the world. In both Canada and the USA, this is motivated by a tax deadline; donations to certain charities by December 31 can be tax deductions for that year. It’s an opportunity to lay out here a concept that I helped draft a decade ago: the “Social Justice Tithe”.
The Social Justice Tithe means giving at least 10% of your income to some combination of charities, religious groups, and political groups that enact your values.
This is a simple tip, but one which uncovers that which Microsoft’s Report Builder 3.0 hides in plain sight. It took me forever to figure out. Here are some screen shots, so that web searchers of the future won’t have to take so long.
Report Builder 3.0 is an application which lets you design reports for use with Microsoft’s SQL Server Reporting Services 2012 (SSRS). One of my clients has me using SSRS to generate reports, which are formatted as PDF files. Obviously this requires a pagination process: laying out report content into PDF pages of specified sizes, and keeping specified margins of blank space around the page’s edge.
I discovered that SSRS defaults to using 1″ margins around a Letter-sized page (8.5″ x 11″). I wanted to change that, to make the margins smaller, and to let me use more of the page. SSRS has pretty good documentation, including a helpful MSDN article Pagination in Reporting Services (Report Builder and SSRS). This article advises,
“By default, the page size is 8.5 x 11 inches but you can change this size by using the Report Properties, Page Setup dialog box or by changing the PageHeight and PageWidth properties in the Properties pane…. Margins are specified using the Report Properties, Page Setup dialog box or by changing the TopMargin, BottomMargin, LeftMargin and RightMargin properties in the Properties pane.”
Great! I clearly wanted to find the TopMargin and BottomMargin properties in the Properties pane. I could see the Properties pane easily (see the first screen shot, click for a full-sized rendition). But I couldn’t find any PageSize or Margin properties. Next, I looked for a way to bring up the Report Properties dialog box. I expected this to be on the Tools or Format or File menu of a Windows app. But Report Builder 3.0 is built with a new UI design, which lacked the traditional Windows menu bar.
So, for a long time I was stuck. I couldn’t find how to make report-level properties like PageSize or Margin appear in the Properties pane. And I couldn’t find the Report Properties dialog box. Nor could I find documentation on how to do these things, just documentation that assumed I could do it. I was stuck for a couple of weeks, living with unsatisfactory margins, and getting other things done. Until I discovered the trick.
Simply put, the trick is to click outside the report body. The second screen shot adds annotations, showing important parts of the Report Builder 3.0 window.
I achieved my goal, making the report margins smaller, by clicking at B. The Margins parameter set appeared in the Properties pane. From there, it was easy to adjust the value of the Top and Bottom Margins parameters.
And now I’ve published this robobait, in hopes of seeding search engines with notes of this trick, and helping others get unstuck faster. Please let me know in the comments if this was useful.
We recently imported our 2005 Mazda 3, built to California standards, into Canada. As part of that, we had to convert it to have daytime running lights — the headlights needed to stay on anytime the car is running. Why? Safety, it is said; a waste of fuel, it is also said; but the main thing is that C.R.C. c. 1038 Schedule IV Standard 108 requires it, and so we did it. It was almost, but not quite, a simple matter of installing a $40 aftermarket controller module (Hamsar 70987) bought from Canadian Tire. Here’s the lessons we learned that weren’t in the instructions. I hope they will help others installing daytime running lights in a Mazda 3.
1. Getting physical access to the headlight wiring is hard. The module requires one wire each to be attached to the active (+) wire of the low-beam lights, high-beam lights, and turn signal lights. These lights are, on the left side, hidden under a (removable) battery compartment vent, plus various framework pieces of the engine mounting and body front. Attaching the module wire to the existing headlight wire isn’t hard; the kit includes splice taps which makes the wire splicing a simple matter of placing wires and squeezing with pliers. But physically reaching the existing wires, between and behind all that structure, was hard.
2. The trick to reaching the turn signal lights is to go up from below, through the front wheel well. The turn signal lights are nestled under a particularly thick tangle of structural members. I thought I would have to take off the headlight assembly to reach the turn signal light wiring. But no; I found a posting on reparing tail lights which explained the brilliant idea of coming up from below. I turned the steering wheel to the left, moving the forward edge of the left front wheel out of the way and opening room in the wheel well. Then I unscrewed the two screws holding the front edge of the (plastic, flexible) wheel well guard to the car body. This allowed me to pull back the wheel well guard, reach my hand up between the guard and the body, and reach the wire harness at the back of the turn signal light without too much difficulty.
3. Mazda 3 spark plug wires don’t reliably trigger this module; you need to attach the wire to a wire with a reliable voltage which turns on only when the car is running. The place to find such a wire is in the wiring patch panel in the middle of the fuze box. The module has a green wire, which controls the module. Put a voltage on this wire (relative to auto-body ground) and the module turns on the headlights. Remove the voltage, and after a few seconds, the headlights go off. The kit suggests wrapping the wire around the spark plug wire, and securing with cable ties. I suppose the idea is that the sparking voltage down the wire will induce a voltage in the wire. I tried this; the result was headlights which were on most of the time, but sagged off during deceleration. This rig failed the import inspection — twice.
Note that Mazda 3 spark plugs are wired with two small-gauge conductors, not one thick conductor. I suspect the electrical properties of the Mazda 3 spark plug wires aren’t what the module required. In any case, the green wire, wrapped around the spark plug wires, wasn’t adequate.
I was not up for determining which wire in the engine had the right voltage at the right time. So I gave up and went to a mechanic. The mechanic, having done a couple hundred such installations, had no trouble finding a wire in the patch panel which had a dependable voltage. Thus, the lights became reliable.
I hope these notes and photos will help other Mazda 3 owners who need to install aftermarket daytime running lights.
I recently had an adventure trying to get a plugin, PDT, installed into my Eclipse software development environment. Diagnosis was hard, and the conclusion was non-obvious, though in hindsight, reasonable. It is that if you want to use an Eclipse plug-in that requires Java SE 6 on a Mac OS X 10.5.8 computer, you will need the 64-bit version of Eclipse.
Let me explain.