A Solution Looking for a Problem

In one of those moments of questionable inspiration that hit me from time to time, I decided to figure out how to take the clippings file on my Kindle and convert it into something easier to browse and manipulate off-line on my PC. As I’m a fairly decent hand with the PHP programming language, that was my obvious tool of choice. Since PHP is designed with web applications in mind, it seemed equally obvious for me to make the tool available on the web, should anyone else desire to do the same thing with their Kindle clippings (Klippings?).

What I came up with is a simple web form via which you can upload your “My Clippings.txt” file from your Kindle, which you can find in the “documents” folder of your Kindle device when you connect it to your computer via USB. Once you submit it to the web page, the program breaks the text up into individual clips, then parses each clip to extract the various data elements. Those data elements are then written to a temporary CSV (comma-separated values) file, and when they are all written the file is sent back to the user as a file download, which can be saved or opened in a compatible program, e.g. pretty much any spreadsheet program. (If your spreadsheet program asks, you should specify comma as the field separator and double quotes as the text, umm…, quoter? Also specify UTF-8 as the character set for best compatibility.)

Once you’ve loaded it into a spreadsheet (or database, if you’re as geeky as I am), you can then sort your highlights and notes by book title, author, date, etc. The default sorting is first by title, then by author, then by location number.

CSV file example

[This was fixed 2011/07/02.] I have found one minor bug: if a location has a page number associated with it, the text gets a bit screwed up (sorry for the technical jargon). If you try it and find any other bugs or have any suggestions, please feel free to leave a comment here.

Lastly, I’ll repeat what it says on the tool’s web page: “Privacy Policy: Your clippings file will not be saved anywhere by this web page. The data is parsed, output is created and sent back to you, and then discarded at this end. None of your data will ever be used by me nor shared with anyone else.”

Jailbreaking Kindle 3.2.1 Firmware

Apparently all new Kindles are now shipping with firmware version 3.2.1 (not just the “sponsored” Kindles). As a result, the jailbreak hack which is needed to install enhancements such as custom screen-saver images or alternative fonts will not work. However, a new jailbreak version has been developed which gets around this, with the downside being that its implementation is somewhat trickier than the earlier version.

If you have a 3.2.1 version Kindle and want to jailbreak it, go to Yifan Lu’s blog post, download the file, and read and follow the instructions carefully.

As always, you assume full responsibility and risk for altering your Kindle, which may void your warranty. (I have not heard of Amazon refusing to honor a warranty yet for such things, but I suspect if you hack a “sponsored” Kindle in order to eliminate the advertisement screen-savers, they may be less friendly about it than they have been — but that is only speculation on my part.)

Dithering your Kindle Screen-Saver Images

What the heck is dithering? Wikipedia says, “Dither is an intentionally applied form of noise used to randomize quantization error, preventing large-scale patterns such as ‘banding’ in images.” The only reason you probably need to care about that is if you’d like to create your own images to use on your Kindle as a screen-saver (a.k.a. sleep screen), or if you are a content provider who would like to optimize your images for the Kindle.

The reason you may want to dither an image is because the Kindle e-ink screen only displays 16 shades of gray. If it tries to display an image that has gradual changes of darkness/lightness, you will likely end up seeing distinct bands of the different grays that are available, instead of a seamless transition. As an example, let’s suppose we start with an extreme example:

If we display it on the Kindle as is (or even if it is converted to gray-scale first), it will end up looking something like this on the Kindle’s screen:

What I have found to be a good solution when working with the GIMP image editor is to first change it to an “indexed” image (using the Image -> Mode -> Indexed… menu option). In the resulting pop-up, I select the option to “Use custom palette” and select a 16-shade palette I created for this purpose. Then in the “Dithering” section of that pop-up I select “Floyd-Steinberg (normal)”, then click the Convert button. This would convert our original sample image to:

If you were to zoom in that image, you would see that it is not the apparent smooth transition from one shade of gray to another, but that there are a very limited number of actual shades per pixel, but they have been pseudo-randomly blended together to give the smooth effect:

Once you’ve done this dithering by changing the image mode to an indexed palette image, you then want to once again change the mode to grayscale (Image -> Mode -> Grayscale) before saving it as a PNG file. This final mode change is needed because, at least in my experience, the way the Kindle displays palette-based PNG images is unpredictable and less than optimal, but seems to work fine when it is saved as a grayscale PNG, instead.

If you would like to try this and are a GIMP user (it’s a free, open source product, which is why I use it), it comes “out of the box” with a 32-color gray palette which will help smooth things out, but you can use my custom 16-color gray palette by downloading it, unzipping it, and saving it in your custom palettes directory. (Go to Edit -> Preferences from the menu bar, then in the new window find the “Folders” section on the left, and drill down into it to find and select “Palettes”. The main part of that new window should then tell you what folder it is using for custom palettes, which is where you would want to save this new custom palette file — probably something like “C:\Users\YourName\.gimp-2.6\palettes” if you’re on Windows.)

Is This Why We Have to Hack the Kindle Screen-Saver?

While mowing the lawn today and not really thinking of anything except how much I hate doing yard work, a thought managed to ambush me: is the reason that Amazon has not provided a built-in ability to change the screen-saver (a.k.a. sleep screen) images on the Kindle 2/3/DX because they were thinking all along about the “Special Offers” Kindles with advertisements on those images? (Now I’m thinking: is that the longest single sentence I’ve entered into this blog?)

For at least a couple years now the smart folks who came up with the screen-saver hacks have been able to provide this functionality. When I check this blog’s stats for most common search terms/phrases that brought people here, “screen-saver” (or some permutation) shows up the most by quite a large margin. In all that time, Amazon has ignored the requests from the multitudes who wanted this feature. Could this have been because they were thinking all along about putting ads on the screen-savers and didn’t want us to be able to have access to the screen-saver images?

Maybe I’m giving them too much credit, but everybody loves a good conspiracy theory, right?

Add Justification Option to Kindle’s “Aa” Menu

Kindle keyboard Thanks to a thread at the MobileRead.com forums, I’ve learned that you can add a menu item to the font-size/page-orientation menu accessed via the “Aa” button on the Kindle keyboard. This menu adds the ability to select between full justification (lines are spaced as needed to make both the left and right margins even) or left justification (no spacing is added, resulting in a “ragged right” margin). This is not a hack, but a built-in capability which for whatever reason the Amazon Kindle development team decided not to include in the default menu. (If anyone knows why, I’d like to know.)

Aa menu

In order to enable this menu option, you just need to edit a text file on your Kindle. (Note that you may need to enable the display of “hidden” files on your computer’s file system viewer before you can find this file.) UPDATE: Before connecting your Kindle to your PC, return your Kindle to the Home screen (i.e. close any book you’re currently reading), or else the following editing will not take effect. (Thanks Donna!) The file you want to edit is system/com.amazon.ebook.booklet.reader/reader.pref. (If you’re not particularly geeky, that means after connecting your Kindle to your PC via the USB cable, locate and open the system folder on the Kindle, then open the com.amazon.ebook.booklet.reader folder in there, which should then contain the reader.pref file.)

IMPORTANT: When you edit the reader.pref file, use a text editor (NOT a word processor) which is “UNIX line-ending aware.” (If you use Windows Notepad, it will insert Windows line-endings instead of the UNIX line-endings.) If not sure, I’d recommend using Notepad++, a free download (and not to be confused at all with the Windows Notepad editor). Also, be sure to make a backup copy of the reader.pref file somewhere, just in case you screw things up.

Once you’ve opened the reader.pref file in your text editor (a final reminder, don’t use Windows Notepad or Word!), just add this as a new line:


Editor image

Now all that remains is to eject your Kindle from your PC, then restart it (Home -> Menu -> Settings -> Menu -> Restart). Once it has completed the restart and you’re back to the Home screen, open up a book, and verify that you now have the “Justification” item in the menu when you press the “Aa” button. Here are a couple screen shots from my Kindle 3 showing the same page with full and then left justification (click the images to see larger versions):

justified left justified

Kindle Screen-Saver Image Customizer

I created a little web app that allows you to create a 600-pixel by 800-pixel image suitable for use as a screen-saver on your Kindle 2 or Kindle 3, if you have installed the screen-saver hack on your K2 or K3. (At this time I’m not supporting larger images for the DX — maybe in the future?) Just point your browser to www.ebookworm.us/screensaver, select the image you would like to use, enter your name if you would like it to have it displayed as “Name’s kindle” (see image below), and also optionally add text to display at the bottom of the image. (I recommend an email address and/or phone number where someone can contact you if they find your Kindle.)

At the time of this writing there are three images to choose from, but I’ll probably add a few more over the next week or so as I come across suitable (and royalty-free) pictures.

If you run into any bugs, post a comment here to let me know.

Jail-Breaking Kindle 3.1 Firmware

Based on info I’d seen on various web forums, I uninstalled the screen-saver hack and the jail-break hack (the latter is needed in order for the former to be installed) on my Kindle 3 before I installed the 3.1 version firmware version from Amazon. Now there is a 3.1-compatible jail-break available.

I took the plunge tonight and downloaded and installed it, which seemed to go smoothly (no error messages of any sort). I then installed the same screen-saver hack I’d had on my K3 before (version 0.18.N), and it also installed smoothly. I powered off my Kindle, and my personalized screen-savers were back just like before. (Since I had not deleted the screen-saver directory the original hacking had created, it apparently re-used them this time around.)

Now I’m a (relatively) happy camper, no longer having to see the infamous “dead authors” when I turn off my Kindle.

Onward and Upward to 3.1 for Me

I decided to download and install the preview version of the 3.1 version of the Kindle firmware tonight. I had no special reason to do so, other than a bit of time on my hands — oh, and so I could report on it to my gentle readers, of course.

The news I’ve read in web forums indicates that once 3.1 is installed, any of the “jailbreak”-related hacks will not be removable, at least until the smart people who came up with them figure out how to bypass whatever the Amazon developers changed that affects these hacks. So first I uninstalled my screen-saver hack (the only hack I had installed) and then the jailbreak hack. I then followed Amazon’s instructions, downloading the appropriate patch file for my WiFi-only Kindle 3, saving it in the top-level directory on my Kindle, then ejecting it and selecting the “Update” option from the settings menu.

All is good to go and my K3 shows it’s now at software version 3.1. I’ll play around with it for a day or two, then get back to you here with anything interesting or otherwise noteworthy about it (or at least anything that will fill out another blog article).

PS (2011/02/18): It appears Amazon is already rolling out this update via automated downloads/updates, so you may want to uninstall any hacks ASAP if you are concerned about possibly not being able to uninstall them later.

Updates to Screen-Saver and Font Hacks

Just a quick note to let you know that “NiLuJe” has released updates to the Kindle font and screen-saver “hacks” today. Be sure to check out the change log at the MobileRead.com forum post for what has changed, and the related Wiki articles for the font installation and screen-saver installation instructions.

The font changes are:

  • A tiny code cleanup.
  • Support light FT hinting (override)
  • Fix the FW 3.x browser when using an FT override.

The screen-saver changes are:

  • Fixed a confusing install log message
  • Updated sort binary (updated coreutils)

So I don’t think most of us need to worry about updating if already successfully running an earlier version.

Kindle 3 Font Hack Now Available

“NiLuJe” has released a Kindle 3 compatible font hack today. So in addition to changing the screen-saver images, you can now also change to some different typefaces if you would like. See the MobileRead Wiki instructions for installing and using this hack. As in all such cases, you are solely responsible for modifying your Kindle in this way. It is not an approved Amazon update. Here are the Zip files (separate versions for Kindle 2 and for Kindle 3):

For samples of many of the available fonts, see my font samples page. I’m a Georgia2 fan (K2 screenshot):

Georgia2 sample