Linux Search & Kill Group of Processes

The following quick command can be used to kill one or more processes found using the combination of ps and grep.

ps -ef | grep FooBar | awk '{print $2}' | xargs kill -9

Ruby Error on Require JSON in Ubuntu

Received the following error from a Ruby application when it called require 'json'.

irb(main):002:0> require 'rubygems'
=> true
irb(main):004:0> require 'json'
LoadError: no such file to load -- json
from /usr/lib/ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require'
from /usr/lib/ruby/1.8/rubygems/custom_require.rb:31:in `require'
from (irb):4
from :0

JSON is in the libjson-ruby package. So to fix this

sudo apt-get install libjson-ruby

and presto!

irb(main):001:0> require 'json'
=> true

Linux Argument List Too Long

Encountered this cleaning out a directory overflowing with thousands of log files. When entering rm *.log or even a more narrow set such as rm *2011.log, the following error occurs.

sudo: unable to execute /bin/rm: Argument list too long

Solutions

Use find to output the file names one at a time, sending each to the rm (or some other) command.

find /home/username -type f -name "*.log" | xargs rm -f

find /home/username -maxdepth 1 -type f -name "*.log" | xargs rm -f

find $HOME -maxdepth 1 -type f -name "*.log" -exec rm "{}" \;

Django Notes

A few tips, tricks, gotcha's, and other findings from working with Django for Python.

Complex Queries

Complex queries are run in Django as "raw" queries. The following information will get you well on your way with complex queries including queries on databases other than the default in a multi-database setup.

• Multiple databases require Django 1.2+
• When performing raw queries, the table and column names are the actual names define in the database, not the names modeled in Django.
• Raw query must include the primary key
• Use db_manager('mydb') to specify other than the default database.
  Ex: Author.objects.db_manager('mydb').raw('SELECT id, name FROM library_authors WHERE ....')
• Use cursor for even more complex queries. These can be handled by a class to simplify accessing.
• Use 2 %'s (%%) when using % in a raw query.
  Ex: "LIKE %foo" ~~> "LIKE %%foo"

References

https://docs.djangoproject.com/en/dev/topics/db/multi-db/#using-raw-cursors-with-multiple-databases
https://docs.djangoproject.com/en/1.3/topics/db/sql/
http://osdir.com/ml/django-users/2010-01/msg01419.html
http://www.djangobook.com/en/2.0/chapter10/
https://docs.djangoproject.com/en/dev/topics/db/queries/#escaping-percent-signs-and-underscores-in-like-statements

Note the warning about passing parameters to raw,
https://docs.djangoproject.com/en/dev/topics/db/sql/#passing-parameters-into-raw

Modeling Existing Tables

When modeling existing tables by default, Django gives each model an "id" field. This can cause an unknown column error if your table does not have an id field. Resolve this by declaring another column as the primary_key.

see:
https://docs.djangoproject.com/en/dev/topics/db/models/#automatic-primary-key-fields

Passing Data to a Template

• Use safe to bypass Django's automagic HTML special character escaping.
  Reference: https://docs.djangoproject.com/en/1.2/topics/templates/#automatic-html-escaping
  {{ myVar|safe }}

Troubleshooting

Caught OperationalError while rendering: (1054, "Unknown column 'myTable.id' in 'field list'")

https://docs.djangoproject.com/en/dev/topics/db/models/#automatic-primary-key-fields
must assign a column as primary key so Django does not try to use default 'id'

Cool Quick Linux Tools

These cool tools are awesome for tweeking your Linux workflow. Admin multiple machines at the same time, add GUI dialogs to your bash scripts, and more.

In Short: shFlags, xvkbd, xclip, Zenity, cssh, tree

Zenity Date Select Dialog

Let's start with the one utility, shFlags, that would be useful in any shell script. shFlags (Shell Flags) is a terrific library that simplifies handling of command-line flags, parameters, and arguments passed to your script. It aims to be a portable cross shell/OS alternative to getopt. In the tests I've done, shFlags rocks.

xvkbd is the X Virtual Keyboard and, as it sounds, is a virtual keyboard for the X window system. It can be useful for kiosk installations, however you can use its command-line functionality to send commands to X windows from within a script. The script can fill data in a window, nice.

xclip provides a command line interface to the X11 clipboard. Hugh? Without touching the mouse, you can copy data that can be later pasted in X. A script could process some data and put the result in the clipboard for the user. xclip's functionality does not end there, but I will leave that for you to discover.

Zenity, the key to providing GTK dialog boxes for shell scripts. Confirm decisions, get input, show progress all in a nifty GTK dialog called from within the shell script. The short examples on the website obviate how cool Zenity is.

ClusterSSH, available in most Linux repositories or downloadable from Sourceforge, allows you to ssh into and issue commands to multiple machines at the same time. Now making quick modifications on multiple clients is a breeze.

Tree. This seemingly simple command line tool is amazing for displaying your directory structure in a tree format. Visualize and analyze your directory and file structure quickly and easily.

Linux Timezone QuickTip

Recently, after moving to a different timezone, I noticed that my automated backup scripts were failing. Why? They were running based on the old timezone (during a disallowed time).

To update to the new timezone, specify the correct timezone in the directory path below:

cd /etc

sudo ln -sf /usr/share/zoneinfo/US/[yourzone] localtime

Issue the date command to see the new date/time/zone.

Logic:
/etc/localtime is a symbolic link to the correct timezone data as stored in /usr/share/zoneinfo.  Directories are also available for other countries within the zoneinfo directory.

Google Tools (I Didn't Know I Was Missing)

My Google search results recently returned a couple of Google tools that I hadn't seen listed yet. I occasionally browse the more | even more page in my Google account to see what's new. Here are two tools I haven't seen listed there yet (one very useful, one just for fun).

• Google Financial Comparison
  Find and compare rates on mortgages, credit cards, CDs, checking and savings accounts.
  
  Includes a nice overview, side-by-side comparison, and more. And in typical Google style, all without being bombarded with obnoxious advertising like many other bank/financial rating sites.
  
• Google Hotpot
  Rate and review places you know, see recommendations from friends and find new places within your profile based on the ratings and reviews you submit.

I Need Big Expensive Software! (Not Really)

A recent issue of Database Trends and Applications magazine quoted a Forrester Research study that found 80% of database installations only use 20% of the features provided. My estimation is that this statistic also applies to other software, especially some of the most "popular" used and required on home, school, and office PCs. And this is a large aspect of why open source software should be used, encouraged, and by no means discriminated against in the public arena.

Let's avoid any anti-Windows or anti-Microsoft propaganda or personal preference. Anyone who knows me, already knows my personal preference for Ubuntu Linux anyhow. I'm going to focus briefly on the MS Office suite of applications, because as we know it is the dominant software in this arena.

My wife does things with MS Office at work that would make most people's head spin. She needs and uses the power, APIs, and scripting that it provides. She justifies the $X for her corporate license for Access, Word, Excel, etc. At home, she uses a word processor, rarely, for basic documents and a spreadsheet for rosters with name, address, etc. There is no justifying the expense of MS Office for home. She will occasionally use the old copy of Office97 from our college days, but our primary productivity software is OpenOffice.org. For me, running a Linux desktop, it's always OpenOffice or other open software.

If a corporation decides to buy a corporate license for MS Office or other proprietary productivity software, that's fine by me. When a government spends tax money on licenses for employees who don't use/need the added functionality, it's misuse of resources.

When a public school requires students to use MS Office to read an assignment downloaded from the Internet or as the only format for submitted assignments, there is a problem and an unfair bias toward students who can afford home computers and more over the MS Office software. This irks me. Of course OpenOffice can read/write to MS Office formats, so those "in the know" are OK (most of the time) but the majority of educators do not know this and if they do it is not taught or explained to students.

Teach students fundamentals, not memorized specifics, so that they can rapidly adapt and learn a variety of tools. I have seen countless times educators and students alike turn aside from alternative software options because it's "not the same," "too different," "too hard." The fundamental differences in MS Office and OpenOffice are negligible, and in my opinion, no more difficult to grasp than when MS Office dropped the menu bar and got all decorated with ribbons and bows. Users adapted to the changes due to necessity. It is no more difficult to adapt to another office suite.

The principle of cost vs features vs needed features vs compatibility, should be looked at in all organizations from small to large corporations, schools, governments, churches, and non-profits.

My point in all this is use the software you need for the job your performing and don't force others into your boat when another (potentially free) option will work equally well.

Mounting USB Storage Devices

Today I noticed that my USB storage device (thumb drive, usb drive, memory stick; call it what you will) was not mounting when plugged into the PC. Note: I'm currently running Ubuntu 10.10 Maverick Meerkat.

The solution that fixed this was

sudo apt-get install usbmount

It's also good to note that lsusb will list usb devices detected on your system. Here you'll likely see reference to the usb drive although it isn't being mounted.

And for fun, gconf-editor lets you tweek all manner of settings, one of which is the Nautilus applications setting for auto mounting storage devices.


Note:
usbmount has been known to contain a bug that causes the device to mount as root instead of the logged in user, meaning you cannot modify the data. If this is the case usbmount may not be the correct fix for you. You can also try using Disk Utility to unmount/mount the device and see if that works.
     System | Administration | Disk Utility

Removing Partition, Grub, & Fixing MBR

Unfortunately had to remove Linux from a dual boot machine today due to compatibility issues (shame on you hardware manufacturers). I wanted to reassign the Linux partition space back into the Windows partition. No problem, right?  Maybe if you do this type of thing regularly and already have the tools at hand. This is the first time I have actually had to remove Linux, a sad day for sure, but a new set of tools for the future. Following is a brief walkthrough. (note: this was complicated by being a Eee PC meaning no CDs, USB memory devices only.)

In Brief:

• Parted Magic Live USB
• remove Linux partition
• wipe MBR from terminal
  dd if=/dev/zero of=/dev/sda bs=446 count=1
• Windows/Dos Bootable USB
• xfdisk /MBR

Details of My Steps:

• Use Parted Magic to remove the Linux partition and reassign space to Windows.
• http://partedmagic.com/download.html  (see 'partedMagic' from USB section)
• install toUSB instructions http://partedmagic.com/documentation/130-creating-the-liveusb.html
• This will make GRUB cranky (Grub error 22: partition deleted). Wipe the MBR with the exception of the partition table (hence 446 below).
• while still in Parted Magic launch a terminal and enter the command
  dd if=/dev/zero of=/dev/sda bs=446 count=1
• as per http://www.linuxquestions.org/questions/linux-hardware-18/how-to-remove-grub-using-partedmagic-789555/#post3866148
• Now the machine just won't boot, so we need a boot disk to fix the MBR.
• using the following instructions, create a basic windows boot USB (this was the quickest and easiest method I saw)
  http://forums.xbox-scene.com/index.php?showtopic=630799
• Ultimate Boot CD contains the xfdisk tool needed to fix the MBR. You can probably find xfdisk elsewhere, but since I already had UBCD... copy the xfdisk directory from UBCD to your new USB boot disk.
• Boot the PC using the new boot USB
• at the command prompt change to the xfdisk directory
• issue the command xfdisk.exe /MBR
• Remove the boot USB, reboot and all should be well.

Alternatively:

You should also be able to do roughly the same using the Windows Recovery Console to issue a fixmbr, fixboot, fdisk /MBR, fixboot c:, or bootcfg /rebuild.  In my case I did the above steps while waiting for files to copy to USB so I could access the recovery console. Instructions for getting a USB bootable recovery console can be found at http://www.msfn.org/board/USB-Boot-Recovery-Console-t111986.html although I never completed the steps and cannot confirm their functionality.