Last time I covered the basic use of dd, Linux's "Convert and copy a file" command. Here are a few more things you can do with it, some trivial, some useful.



File Conversion
Convert a file to uppercase:

dd if=filename of=filename conv=ucase

Convert a file to lowercase:

dd if=filename of=filename conv=lcase

(Incidentally, if you leave out of= output gets written to "standard output" -- which defaults to the screen.)



File rescue
If a video or sound clip won't play because of bad blocks, copy it, telling dd to ignore the corrupt stuff

dd if=badfile.wav of=fixedfile.wav conv=noerror

Wipe all the free space on a partition
(ie: Deleted files you don't want recovered)

dd if=/dev/urandom of=~/trash.file

This will create a file called trash.file on your home partition and fill it with random characters. Eventually dd wil stop with the message No room left on device meaning all free space has been consumed. Then simply delete trash.file with:

rm trash.file

Note that dd only overwrites, it doesn't provide a cryptographic deletion. For that you're better off with srm and sfill. (See here.)



Look at your disk
If you have the program hexdump installed (it's available in most standard repositories), you can check out the contents of your hard drive.

Look at the first sector on your hard drive:

dd if=/dev/sda count=1 | hexdump -C

dd defaults to a byte count (bs=) of 512, so this will show the first 512 bytes on your disk. (The first 446 are the MBR, the remainder are the partition table.)

Posted by Geoff Palmer at 9:37 PM | Permalink | Comments ( 1 )

Despite its dull-sounding description -- "Convert and copy a file" -- dd is one of the most powerful tools in the Linux toolbox. That's because in Linux, everything is a file! I should warn you however that dd can also be d-d-dangerous if you get your ifs and ofs mixed up. But more on that in a moment.

To get a feel for what it can do, imagine you have a USB drive you want to temporarily use for something else. Running df -h shows it's mounted on /dev/sdc. The commmand...

dd if=/dev/sdc of=/path/to/image bs=4096

...will do just that. To write the image back to the drive you'd use...

dd if=/path/to/image of=/dev/sdc bs=4096

Notice how similar those commands are? We've just swapped the if= and the of=. That's what makes dd dangerous. If you mixed them up, in this example you'd write the new contents of your USB disk over the old image file, essentially nuking it. Be careful!

(By convention, if= comes first, but there's nothing to stop you using dd of=/dev/sdc if=/path/to/image bs=4096 for the restore step.)

The bs=4096, by the way, is optional. dd reads and writes 512 bytes at a time by default. Increasing this will speed up writes to the USB drive.

Since we're talking about syntax, here's a brief overview. (For more details type dd --help.)

if=	Input file.
of=	Output file.
bs=	Bytes to read and write at a time. The default is 512.
count=	Copy this number of blocks from the input file (instead  of everything until the end of the file is reached).
conv=	Convert the file as specified in the following comma-separated arguments.
noerror	Continue even if read errors are encountered. (dd will normall stop.)
notrunc	Don't truncate the output file.

Create an ISO image of a CD or DVD

dd if=/dev/cdrom of=/path/to/image.iso bs=2048 conv=sync,notrunc

The ISO image can then be used in a regular CD/DVD burner, or you can mount and access it locally with these commands...

mkdir /mnt/temp mount -o loop /path/to/image.iso /mnt/temp/ cd /mnt/temp

MBR Tricks
Backup the Master Boot Record (MBR) including the partition table:

dd if=/dev/sdx of=/path/to/image count=1 bs=512

Backup the MBR without the partition table:
(Partition tables are unique to each disk so unless you're cloning a whole drive you shouldn't overwrite them.)

dd if=/dev/sdx of=/path/to/image count=1 bs=446

Restore the MBR:

dd if=/path/to/image of=/dev/sdx

Backing Up Drives
dd makes cloning drives easy, but you shouldn't do it with either drive mounted! Boot from a Linux startup CD and run the command from there.

Backup an existing drive:

dd if=/dev/sdx of=/dev/sdy

Backup a hard disk to an image file:

dd if=/dev/sdx of=/path/to/image

Backup a hard disk to a compressed image file:

dd if=/dev/sdx | gzip > /path/to/image.gz

Restore a backup image:

dd if=/path/to/image of=/dev/sdx

Restore a compressed backup image:

gzip -dc /path/to/image.gz | dd of=/dev/sdx

Backup a partition to a different partition on another drive:

dd if=/dev/sdx2 of=/dev/sdy5 bs=4096 conv=notrunc,noerror

Nuking a Disk
You don't have to write meaningful data to a disk, of course. You can use dd to wipe a disk too.

Wipe an entire disk by filling it with random data:

dd if=/dev/urandom of=/dev/sdx conv=notrunc

Wipe an entire disk by filling it with zeros:

dd if=/dev/zero of=/dev/sdx conv=notrunc

There are some even more cunning tricks you can do with dd. More next time!

Posted by Geoff Palmer at 6:57 PM | Permalink | Comments ( 3 )

Way back in the September 2008 issue of our print edition, I warned readers about the cash-back schemes offered by some computer retailers. Cash-backs are still proving a concern, according to the latest issue of Consumer magazine. They report that they've received many complaints from people "struggling through the hoops required to get their cash back". One member waited more than six months for a refund -- which they only received after Consumer got involved.

Little changes.

Two years ago I reported on one cash-back that required:

• Buying a qualifying product between two particular dates.
• Registering an intent to claim online before third (in this case before midnight Australian Eastern Daylight Savings Time‭!)
• Printing out and posting the relevant paperwork to arrive before a fourth date
• In addition to proof of purchase, you had to supply ‬your name,‭ ‬address,‭ ‬contact details and bank account number.
• You had to join the manufacturer's online club.
• You had to confirm you'd read and agreed to the promotion's terms and conditions, including an agreement that you're a New Zealand resident,‭ ‬but not the‭ "‬spouse,‭ ‬defacto spouse,‭ ‬parent,‭ ‬natural or adopted child,‭ ‬and sibling‭ (‬whether natural or adopted by a parent‭) ‬whether or not they live in the same household‭" ‬of one of the manufacturer's employees,‭ ‬contractors,‭ ‬stockists,‭ ‬resellers,‭ ‬dealers,‭ ‬distributors or retailers.
And you further agreed that:
• No payouts would be made until 4 to 6 weeks after the closing date.
• The promotion could be cancelled at any time for any reason and pay you nothing
• And that‭ ‬though the promoter would hold your personal and financial details for up to six months after the promotion closed,‭ ‬they accepted no responsibility for its loss or unauthorised access.

All that for payout of between $30 and $50.

And it's still going on.

Consumer report that the Commerce Commission are "cautioning retailers selling computers, and computer manufacturers, that they are pushing the boundaries of the Fair Trading Act." In short, they're close to misleading consumers about products services.

Consumer's advice: "Before you consider a cash-back offer, ask yourself if it's really worth hassle."

And mine, from 2008: "‭Items are discounted for a reason,‭ ‬and in the fast-changing computer business that reason is often to make way for new stock and ditch unpopular or end-of-line models.‭ ‬So when it comes to cash-backs,‭ ‬caveat emptor‭!"

Posted by Geoff Palmer at 7:35 PM | Permalink | Comments ( 1 )

The Gimp Animation Package -- or GAP for short -- is a plugin for Gimp, the free graphics package for Linux, Mac and Windows (downloads here). Rather than add yet another tutorial in its use, here are some examples of it in action. Just follow the links to see how they were done.

Falling snow		Flowing water		Animations inside animations
Kaleidoscope		Matrix effect		Melting

Flames



Simple text animation



Image rotator



Pixel stretch

 

Flashing text  



Thumping image



Border highlight



Incorporating video



Rain


Fluttering flag

You can even use GAP to create animated DVD menus. You'll find a sample file here.

Posted by Geoff Palmer at 4:16 PM | Permalink | Comments ( 2 )

Kubuntu 10.4 ("Lucid Lynx") hit the servers late last week so I thought I'd upgrade. Plenty of people appear to have had a great upgrade experience. Mine was a nightmare. But few Linux disasters are unrecoverable. Here's how I got through mine ... albeit with a few outstanding issues.

I kicked off the Distribution Upgrade process from kpackagekit on Friday evening and left the machine to it. When I checked it on Saturday morning, everything was finished, there were no warnings on the screen, so I rebooted. In retrospect I should perhaps have checked the notifier for any outstanding problems. But surely, anything major would warrant a big bold warning box ...

First problem: a dead monitor. In place of the usual boot menu I got a monitor message: "Out of range: 74.6KHz/60.0Hz". Well, no real problem. That's just a Grub2 setting. After time-out the boot started and text messages started scrolling up the screen.

Second problem: the GUI session starts with the message, "Ubuntu is running in low-graphics mode ... OK". Again, no real problem as once I get into the system I can fix that (it's probably the NVidia driver). But that led to the ...

Killer problem: neither the keyboard nor mouse would respond. That makes fixing things somewhat more difficult.

At first I thought it might be a USB problem so I tried a PS/2 keyboard and mouse. Nope. A few booting experiments showed that it was definitely Linux as the DEL key still took me into BIOS. It wasn't until that "Out of range" message that contact was lost -- a fact proved by the non-responsiveness of Ctrl+Alt+F2, F3, etc. as the messages scrolled.

There was nothing for it but to whip out my trusty copy of SysRescueCD, a brilliant boot disc that's saved mine and many other peoples' bacon over the years. Using it I was able to chroot into Kubuntu with the command chroot /dev/sda1 /bin/bash.

For the uninitiated, chroot changes to the root directory of your target and allows you to execute commands there as the root user -- which in Linuxland equals god. So although I'd booted and was running from a CD, chroot took me into my Kubuntu / partition and allowed me to work there as though I'd managed to boot into it.

Once there I quickly discovered the upgrade hadn't worked fully. apt-get update (no need for sudo cos this really is god mode!), told me there were errors and that I should run dpkg --configure -a. That fixed a lot of errors. A re-run of apt-get update followed by an apt-get upgrade uploaded 20 more missing packages, but there were some it couldn't handle:

Errors were encountered while processing:
 memtest86+
 linux-image-2.6.32-21-generic
 ubuntu-standard
 winbind
 linux-image-generic
 nvidia-current
 nvidia-185-libvdpau
 libglade2.0-cil
 nvidia-glx-185
 linux-generic
 nvidia-185-kernel-source

What's more there were another 432 it still couldn't upgrade!

Hmmm, well perhaps in Kubuntu proper ...

(While I was chrooting around I also commented out the GRUB_GFXMODE=1920x1080 line in /etc/default/grub and tried running update-grub. That wouldn't work for some reason, but I was impatient to try rebooting so put up with the "Out of range" screen once again.)

A reboot got me a little further; right into KDE in fact -- in 640x480 mode -- but still with no keyboard or mouse. I tried a few times, hitting the PC's reboot button each time then finally decided to just shut the damn thing off and have a think. That time I hit the Power button. KDE responded with its "Turning off in 30 seconds" message, and then a miracle happened. Instead of turning off, it dropped back into text mode where the keyboard worked once again!

I hastily did an sudo apt-get update then a sudo apt-get upgrade which fixed all of the above errors, then followed it with a sudo apt-get dist-upgrade. The download process started and took ... five ... more ... hours!! I'm guessing that was because of the demand on Kubuntu's servers. Normally, a complete distribution CD takes 10-15 minutes on my cable connection.

Finally, five hours later, a visible reboot and something like a functional and fully accessible system. The menu had vanished, necessitating the addition of a new Panel and all the widgets it typically contains -- Menu, Pager, System Tray, Task Manager, Device Notifier, Digital Clock, Trashcan and, because of my lycanthropy, Luna -- but otherwise all is well. Well almost.

I now have to boot the system twice to get in. The first boot stops at a line of text reading something like "214683 pages non-shared". And that's it. Lock-up. But the next boot works fine.

The other problem is totally weird. OpenOffice 3.2 starts in seconds, but opening a document takes twenty. I'm not kidding. Twenty seconds! I've timed it by the desktop clock. WTF?

I'll look at both problems in more detail in the coming days, (any suggestions welcome!). For now, I'm just happy my nightmare's ended.

Posted by Geoff Palmer at 10:09 PM | Permalink | Comments ( 37 )