Monday, July 21, 2014

Linux poem: btrfs

This was an entry for the contest mentioned in the Editor's Note of "Linux Poetry Explains the Kernel, Line By Line". It actually won, probably because it was the only entry :-) It was published on, but the post was removed. So I'm posting it here for posterity.


They call her "Butterface"
And she's a bit experimental; a bit unstable
But she's self-healing; she's getting better
And the boys all say she's the future.

Born of the Oracle
She was made for the enterprising.
She puts up with all your faults
And grows along with you.

Everything, for her, hangs on a tree.
She says a B-tree is the place to be
For fast searching and inserting;
And hey, Butterface likes 'em BIG.

She may be a COW
But that's a good thing;
She'll never lock you out,
Or make you wait your turn.

Oh, Butterface, will you be mine?
Will you keep my heart safe
When it's all on the line?
Will you span disks for me?
Will you work in log time?

Oh, my Butterface,
You can defrag me anytime.
I want to administer you
Because I know you're so fine.

Note: I wanted to write a poem about something kernel-related, since Linux really is the kernel, not the GNU OS that usually sits on top of it. File systems are built into the kernel, so that seemed like a good place to start.

Monday, May 06, 2013

Three Things That Health Care IT Can Learn From Open Source

Why I care

I have been involved in the open source world for several years as a hobbyist. But in my paid work, I have spent the last year immersed in the development of a health care system that is trying to comply with standards and be interoperable. Along the way, I’ve noticed several things that I feel that health care IT is not doing as well as it could. Health care is an important, even central part of our lives, and it seems to me that making health care IT as good as it can be should be a priority.

Here are three open source principles that I think would make health care IT better.

1.    Don’t fork; push it upstream

Terms like “forking” and “upstream” are fairly common in the open source world. The origin of a project, the original developer or development team, is considered to be “upstream” of everyone who uses that project in their own projects. When someone wants to modify a project for their own uses, they are generally encouraged to try to “push the changes upstream,” that is, try to get the original developer to add the changes to the original project, so that everyone can have them. The other option, “forking,” is generally discouraged. If you fork a project, you are taking it, modifying it for your own use, and then keeping the changed version as a separate project. This means that any changes are kept away from the upstream project, and if changes are made upstream, they are not likely to be incorporated into the fork. Forking usually happens when there are problems between the upstream project and a downstream developer, or if the upstream project has become inactive.
In health care IT, forking seems to be the norm. There is a general feeling that the upstream projects are not responsive to input and are very slow to change or improve, and so developers tend to make local changes to the standards rather than improving the standards themselves. This isn’t limited to individual companies, either. There tends to be regional and national versions of the standards that are full of changes that should have been pushed upstream. These changes have little or no relation to region-specific situations; rather, they tend to be local workarounds to limitations discovered in the upstream standards. Others create competing standards that claim to fix problems with the current standards, and then lobby the standards bodies to adopt their standards instead.
Healthcare IT could greatly benefit from the open source model of pushing changes upstream instead of forking.

2.    Build a stack; use modules and plugins

Perhaps because it grew out of the Unix world, open source is all about building a solution from many small pieces that are stacked and plugged together, rather than trying to build a monolithic solution that encompasses every use case. This is analogous to using interchangeable, snap-together pieces to build a toy, rather than making a single, proprietary mold for each toy. The modular approach allows a great deal of flexibility and allows many different use cases to share a great deal of common code. Also, upstream changes are much easier to implement, as they tend to affect only the users of a particular module, rather than affecting all users of the project.
Health care IT tends to rely heavily on global, proprietary systems and standards. The system or standard is expected to encompass all use cases, and single vendors or standards are expected to manage the whole system. There is very little use of modularity in health care IT. This contributes to the first problem (forking), because any change affects the whole system, rather than a small piece of the system. So if a particular use-case is not included in the existing system or standard, a whole new version of the system or standard needs to be created to handle the new use case, rather than creating a new module to “plug in” to the existing system.

3.    Share your work

One of the fundamental principles of open source, perhaps the most important principle, is the belief that developers should share their code. From the licenses that require users of the code to keep it open, to the open repositories where the code is stored, open source developers passionately believe that code should be free, and open, shared code benefits everyone in the end.
Historically, health care IT has been very reluctant to open up their systems and their standards. Often using patient privacy as an excuse, vendors have charged high prices for “interfaces” that expose small pieces of their data to other systems. Standards bodies have put their standards behind paywalls and have made it clear that users of these standards need to pay to license the technology. Because health care systems and standards tend to be monolithic rather than modular, vendors can control the entire system, and demand high prices for any changes.

Going Forward

Large, profitable companies, like Samsung or IBM, that have begun participating in open source have discovered that following open source practices, like pushing upstream, building modularly, and sharing their work may not be the obvious way to be profitable, but it quickly becomes evident that open standards and systems benefit the entire industry, which in turn benefits the individual vendors. Open standards and systems encourage competition and innovation, which in turn improve the quality of all the systems and standards. This is the kind of upward spiral that health care desperately needs.
The ship of health care IT is big, bulky, and hard to turn, but I believe it desperately needs to change course. If the industry cannot change course, the alternative is frightening. There are many icebergs looming in the dark.

Labels: , , , ,

Wednesday, December 21, 2011

From 1080p MKV to 720p MP4

I started with a 13 GB MKV with 1080p h264 video and DTS 5.1 sound. This was far too overwhelming for my current media centre. So I decided to transform and transcode the file into an mp4 with 720p video and stereo AAC audio.

I did this in Ubuntu.

I couldn't simply use avidemux to transcode the video and audio (as I often have done), perhaps because this MKV was not standard enough to be compatible, perhaps because the MKV was so huge, I don't know exactly. So I took a more circuitous route.

First step, tear apart the MKV into its component parts (elementary streams). The odd file extensions are useful to make it clear that these are containerless streams, that really can't be played by (most) video players.

The H264 video first:
mkvextract tracks 1:video.h264

Then the DTS audio:
mkvextract tracks 2:audio.dts

Second step, put the h264 video into an MP4 container.

mp4creator -create=video.h264 -rate=23.976 -use64bits

I found that I needed the -use64bits parameter to get mp4creator to handle the huge file size. The -rate=23.976 was the frame rate of the video, found using VLC.

Third step, use avidemux to crop, resize and transcode the video. As with many "1080p" videos, the width was 1920 but the height was only 880, and a few rows of pixels at the bottom were duplicated. I cropped the sides and the bottom first to get 1554x874, approximately 16:9 aspect ratio (yes, I lost quite a bit on the sides, but I prefer a full screen experience). I got these dimensions by opening a calculator and entering 16 / 9 x 874. The 874 was my post-crop height. I figured out how much to crop from the sides by using (1920 - 1554) / 2, which gave 183, and you can't crop odd numbers, so I cropped 182 off of the left and 184 off of the right. Then I resized using MPlayer resize (to 1280 x 720) and Lanzcos3 resize method.

I encoded with x264, single pass, using a very high quality setting, (quantizer 20). I saved to "video.720.mp4". This produced a 5.3 GB file. The same encoding, set to quantizer 26, produced a more "smeary", more noticeably compressed video, but it still looked reasonably good and it was only 1.4 GB. So you can decide what you prefer.

Waiting for this bit is the longest piece of the puzzle. Whoo. Hours.

Fourth step, I used VLC to transcode the raw DTS to stereo AAC in an mp4 container. I used "Convert/Save" in the File menu, and set the profile to "Audio - AAC (MP4)". I made sure "Video" was unchecked in the "Video codec" tab and in the "Audio codec" tab, I set the Codec to "MP4 Audio (AAC)", the bitrate to 192 kb/s, channels to 2, and sample rate to 48000. I set the output file name to "audio.aac.mp4". I hit the Save and Start buttons and waited. This was short in comparison to the video transcode.

Fifth and final step, I put the video and audio back together using ffmpeg.

ffmpeg -i audio.aac.mp4 -i video.720.mp4 -acodec copy -vcodec copy

The final video was 5.5 GB in size, quite large, but so beautiful and sharp! You can still see the film grain. This video not only played on my media PC, it also played very nicely on my HP TouchPad.

Labels: , , , , , , ,

Friday, July 22, 2011

Please Remove Harmattan Platform Security!

This is an open letter to the decision-makers in the Nokia Harmattan project, prompted by Ville Vainio's suggestion.

There is still some time before the release of the N9. Before that release, please disable the platform security framework (MSSF) by default.

When the concept of the Maemo 6 security framework was first outlined to the community in Amsterdam, Maemo was the OS that was going to end up powering all of Nokia's future high-end devices. Maemo 6 was being designed to cater to massive global markets, and so it needed to provide full DRM capabilities.

When Nokia moved to the joint MeeGo project with Intel, they pushed to include the MSSF in the MeeGo 1.2 release.

Many important things have changed since then. Nokia has announced that it is dropping the MeeGo project; the N9 will be the only Harmattan device released by Nokia (and it will be a limited availability release with no announced plans for the UK or North America); there will be no DRM support in Harmattan; and the Intel-led MeeGo project has declined to include the framework in MeeGo 1.2 and announced a "review of meego security strategy".

Now that most (if not all) of the most compelling reasons for platform security in Harmattan have disappeared, the platform security has become, as one community member phrased it, "dead weight". It no longer has a great deal to contribute to the success of the device or the operating system. It has now become more of a hindrance than a benefit to developers and power users, more of a disabler than an enabler.

If Nokia releases the N9 with platform security enabled, it is guaranteed that an open kernel with platform security disabled will be made immediately available. Almost as surely, any power users and developers wishing to use their devices fully will immediately replace the stock kernel with the community kernel. This seems completely out of character for an open-source project, forcing a major fork at release time!

I am not arguing this as an academic exercise, however. I have a very specific example in mind when I make this plea; I have long maintained a Maemo project called Easy Debian which allows common users to install and run desktop Linux applications (like Open Office) on their handheld devices without having to know much about the internal workings of the operating system.

As far as anyone can tell, my project will simply not work with the security framework. It depends upon an image file being mounted on the loop device and then chrooted into. This image file is mounted dynamically and so the hashes cannot be stored in the system. So these files can't be signed. And even if the basic files could be signed, what about people installing any new apps from the Debian repositories?

See this post. The whole thread is very interesting.

When the N9 is released, Harmattan will be the best example on the market of a mobile, open-source operating system. It should also be the best example of an "open" OS for developers. Please don't add unneeded layers of complexity. Please make the threshold for entry as low as possible. Please keep the legacy of "most hackable device" alive in Harmattan!

Please reconsider the platform security!

I think Andrew's  cogent summary below is worth including in the article:

"Symbian was a mass-market OS; Harmattan won't be.

The provision of a secure DRM system makes sense when the platform is your future, with content providers lining up to provide movies-on-demand and for purchase.

Maemo managed fine for years without platsec, and Android's recent malware problems show that having one doesn't prevent users granting permissions to apps which don't need them."

Labels: , , , , , ,

Sunday, March 06, 2011

Making the WRN3500L into a Print Server

The Netgear WRN3500L came to my notice very recently when a local store had it on sale for the same price as the venerable Linksys WRT54GL, my go-to router for stable-as-a-rock installs for many years. The WRN3500L is also a Linux router, but it has all sorts of cool extras; Wireless N, gigabit Ethernet, and coolest of all, a USB port! Sadly, the supplied firmware didn't have any print server capabilities, and I really wanted that feature.

So, I've managed to turn my Netgear WRN3500L router into a print server (while still keeping it as a nice router). I did so using the standard DD-WRT build and an extra package that I downloaded using ipkg, the package-manager for OpenWrt and DD-WRT.

I used the following resources:
The DD-WRT router database and wiki

Read on for my detailed instructions on doing it yourself. And feel free to comment if you have found things I should add or change.

Read more »

Labels: , , , , , , ,

Thursday, November 11, 2010

N900 Meego chroot part 3: polishing the process

In my first post in this series, I walked you through making a Meego chroot image from the raw images provided by Meego. In the second post, I gave you some rough scripts for starting the Meego UI in a Maemo chroot with the assistance of Easy Debian. In this post I'll present my new Meego image with the newest Meego UI and all the pieces installed to run without Easy Debian. I also present new scripts that streamline and improve the process.

The Image
I have posted a new Meego image (lzma compressed) on for you, meego_1_1_b.img.ext3.lzma. This image has been updated with newer components, and so the UI looks a bit different, and I have installed Xephyr and xbindkeys. I have also copied over wmctrl and the keyboard focus binaries. These pieces now make it possible to just use the Meego image to do everything, without opening the Easy Debian image at all.

The Scripts
I have posted new scripts, meegoscripts2.tgz:
  • I have improved the chrootmeego script to use variables, so you can customize how you run Meego.
  • I have added a new syncmeego script that copies over the necessary files from Maemo to Meego. I got most of this script from this Meego wiki page, and I honestly don't know what effect the copied files actually have on the chroot (except the resolv.conf file, which we already knew about).
  • I have enhanced the startmeegoui script, now called startmeegoui3. This script now starts the Xephyr nested X-Server inside Meego, gives it keyboard focus, and then starts the Meego UI. 
  • I have added a new script, gomee (clever name, eh?), that opens the Meego chroot, syncs it, and then starts the Meego UI.

Some interesting bits in the startmeegoui3 script:
R. Burchell was kind enough to point out after my second post that this environment variable makes all Meego apps start with software rendering. Apparently, that was all that stood in the way of getting most of the Meego apps to run.

/usr/bin/mdecorator -software -remote-theme 2>/dev/null &
echo "sleeping..."
sleep 10
echo "...ok now"

/usr/bin/duihome --desktop -software -remote-theme 2>/dev/null &
The bold lines are new. Adding the sleep command seems to fix the strange white band that was appearing across the top of the Meego home screen. I'm not sure why, but it works, so... there you go.

Still To Do

The media apps (photos and videos) do not work yet. More precisely, they work, but they can't find any media. I suspect that someone needs to show us how to start the media indexer to get them working.
The phone app doesn't work. If you try to start it, it complains about ofonod not being started. If ofonod is started, then the phone app just never starts at all.

Labels: , , , , ,

Saturday, November 06, 2010

Running Meego Handset in an N900 chroot with Easy Debian: Part 2

In this post, I show you how to start the Meego Handset 1.1 "desktop" in a Maemo chroot, using Easy Debian and the image file you made using instructions in the last post.


This is a very shaky process. It is in no way ready for use in any way other than "Oh that's cool!"

The animations on the desktop are very slow, and none of the applications seem to work except Fennec thanks to Robin Burchell, the applications all work now. Please update your scripts!

The Prerequisites:
  • You have a working install of Easy Debian on your N900, and you can start the LXDE desktop in a window.
  • You have made the image from the last post, and you have put it in your /home/user/MyDocs folder.


I have written four little scripts that semi-automate the process of starting the Meego UI. You can download the zip file containing the four (updated!) scripts here at You must put these scripts in the /home/user/meego directory, or edit the scripts to look in another directory.

Put the scripts in the MyDocs folder of your N900 (that's where they'll download if you just choose the root "N900" folder when downloading from the N900), and then, at the Maemo terminal prompt on your N900:

~ $ tar xzvf MyDocs/meegoscripts.tgz


~ $ cd meego
~/meego $ ./debmee

Hopefully, that's all you have to do to get the Meego 1.1 desktop to start up on your N900. Read on for more details of what's going on "under the hood."
Read more »

Labels: , , , , , , ,