Category Archives: tech

Technical and how-to articles

Building a new PVR

Posted on by

<Updated Aug 18, 2011 after a successful PVR rollout>

Technology has evolved since the last MythTV PVR I built, as chronicled here.  Here’s the latest techniques and tech that I’ve used to (start) build(ing) my current PVR. I’ll update this article as I go, as there’s been some bumps along the way, so completion of the project has been slower than I anticipated.

Requirements for my new PVR include:

  • Linux operating system for cost and flexibility reasons
  • Quiet! Fan-less operation if at all possible, external power supply ok
  • Small form factor, black case to fit in with my current home theater gear
  • Video capture with MPEG-2 hardware acceleration to help keep the CPU needed as small as possible, in an expansion card format for the most compact physical footprint .. additionally there must be at least two independent tuners
  • Analog tuners, but would be good if they were capable of digital for when I eventually move to digital/HD
  • IR receiver and transmitter capability for easy remote control and ability of the PVR to use my current set-top box as a source (gives me all the cable company movies and channels that are not available via the basic cable connection
  • Ability to schedule at least 10 shows and retain 5 episodes of each show .. also ability to schedule based on show name alone
  • Ability to perform post-recording processing, such as removing commercials or changing formats
  • Should be able to use a pre-packaged distribution for most if not all of the functions .. I know it’s a home-brew, but I’m tired of messing with individual packages, firmware, and custom codes to make it work. Using a distribution package makes it easier to maintain through updates.
  • Want to purchase the parts from the same supplier if possible (ended up using newegg.ca)

Since I already run MythTV, it was an obvious starting point and given I don’t have an affinity to a specific Linux distribution, I looked at Mythbuntu and Mythdora since I’m familiar with and already run both Ubuntu and Fedora distributions.

After downloading the Mythbuntu 10.10 ISO disk image, I discovered I didn’t have my USB DVD drive, so I wanted to create a bootable USB flash disk.  I followed the excellent instructions at https://help.ubuntu.com/community/Installation/FromUSBStick and successfully burned a bootable Mythbuntu disk on a 2GB USB flash disk via a Ubuntu VM running on my MacBook Pro.


Continue reading →

MySQL Notes

Posted on by

MySQL Command Line and Configuration Notes

Drop tables with wildcard:

There are multiple ways to specify MySQL credentials, this is not the best, but simply an example of how to drop tables using a wildcard pattern. In this case, command line history such as .bash_history will store your MySQL username and password plaintext, and an extended process listing will also reveal both username and password. When run from the command line like this, the SQL commands and the credentials are not stored in the MySQL history file (.mysql_history).  On closed (private) systems, the risk is low, especially if you clean up after these maintenance activities by purging the command histories.

mysql -u user -p password database -e "show tables" | grep "table_pattern_to_drop_" | awk '{print "drop table " $1 ";"}' | mysql -u user -p password database

Update WordPress home URL

Posted on by

There are times when moving or copying WordPress blogs from one server to another, the owner may want to update the URL associated with the specific site.

A simple MySQL update can match the WordPress blog to a new site URL:

mysql> select option_value from wp_options where option_name = 'siteurl';

+--------------------------------+
| option_value                   |
+--------------------------------+
| http://www.example.com |
+--------------------------------+
1 row in set (0.00 sec)

mysql> select option_value from wp_options where option_name = 'home';

+--------------------------------+
| option_value                   |
+--------------------------------+
| http://www.example.com |
+--------------------------------+
1 row in set (0.00 sec)

mysql> update wp_options set option_value='http://server.newsite.com' where option_name='siteurl';

Query OK, 1 row affected (0.00 sec)
Rows matched: 1  Changed: 1  Warnings: 0

mysql> update wp_options set option_value='http://server.newsite.com' where option_name='home';

Query OK, 1 row affected (0.00 sec)
Rows matched: 1  Changed: 1  Warnings: 0
 

Epitome of bad software

Posted on by

There is a reason many people loathe Microsoft software. Before you consider flaming me for that statement, I realize all software has flaws, bugs and eventually crashes. In my experience, even if it’s patched and up to date, the following image happens FAR too frequently with Microsoft software.
Microsoft bugs
I don’t recall having the same issues with Concept Draw, even with complex diagrams. Since I’m just tired of having to redo work over again, good-bye Visio, I’ve just purchased your replacement.

Resetting WordPress user passwords

Posted on by

Resetting WordPress 3.0 user passwords can be done directly within MySQL through the following procedure.  This assumes your installation of WordPress stores user passwords in the wp_users table as MD5 hashes and the unique site prefix for all WordPress tables in MySQL is _x.

Connect to the database via your favorite GUI (phpMyAdmin, Navicat) or command line with either the WordPress role account or any other MySQL user account with select and update privileges on the WordPress database:

update wp_x_users set user_pass = MD5('123abc890') where user_login = 'administrator';

This will update the password for user ‘administrator’ to ‘123abc890’.  Once this has completed, either flush the wp_x_users table or exit the tool used to access the database to cause the updates to be committed.  Sign into WordPress with the new password and optionally change the password via the user interface.

Apple Exemplifies Fine Software Engineering

Posted on by

So I’ve been a recent Apple user for a mere eight years, when I purchased my first iBook  running the new OS X (10.1). I’m a fan of the form engineering that goes several steps beyond the basic function engineering that is so prevalent in consumer technology these days. For Apple, it’s not good enough that there’s windows, they have to look good too – like a master craftsman that puts finishing touches on the product rather than just slapping some cheap molding on and calling it done (or Windows).

This is too fine for words.

After working through successively newer notebooks (iBook, PowerBook, MacBook Pro), I have recently upgraded my first gen MacBook Pro to a new uni-body MBP. All the way through the online store (with the complication of being a grad student and navigating the education part of the online store), the process was pretty painless. But the real wow was when my new MBP showed up three weeks ago and I decided to use the Migrate function to just suck the contents of my old MBP to my shiny new uni-body MBP (thanks for the encouragement, Jonathan). I figured since I didn’t have the time or energy to setup another computer from scratch, I would try this migrate feature – with a heavy dose of battle earned skepticism. When I turned on the power on my new MBP, it seamlessly guided me through the setup .. and asked me if I wanted to migrate from an existing Mac or even a TimeMachine backup of a Mac.  I said yes, hooked the old and the new together .. fully expecting this to not end well and have to restart some install process.  Well a little while later, the migrate was done .. I restarted my new MBP (didn’t have to), and it looked exactly like my old MBP. All of my Applications were there. All my documents where there. iTunes was there. iPhoto was there. The positioning of the icons and documents on my desktop was exactly like my old MBP. Wow. A migrate function that actually worked.  Really. All the way.  Ok, well I did have to re-setup my home wireless connection .. for some reason that didn’t seem to come across, but with the totally customized settings I use, I’m not too surprised although it only added about 120 seconds onto my migrate time.

So at the time I’m writing this, Apple has announced the next generation of the MacBook Pro (the Intel i5 and i7 processors).  Since I’ve only had my shiny new uni-body MBP for a week, I call the folks at Apple and speak to a very pleasant customer service rep (send me an email or website message and I’ll forward his name), who not only cheerfully agrees to accept my new MBP back, but helps me order the new generation. They waived the return shipping and any refurbishment fees, as well as the express shipping for the new unit to me.  Gives me his direct line so if the Apple provided UPS return sticker expires before I get the old-new MBP migrated to the new-new MBP, I can call and get a new label. All this (and I ordered a new mouse) and they refunded a net of nearly $900 back to my credit card.

Well, I’ve just finished the migrate from the old-new MBP to my new-new MBP and again, it was seamless. I don’t think I’ll rebuild a new Mac from scratch any more – this is just too fine for words.  So I can get back to my Master’s thesis and life in general, and not worry about the software out there that is half baked or just barely good enough to get by .. with lots of manual care and feeding.

Thanks Steve and crew – this is why I’m an Apple shareholder.

Accessing Ubuntu desktop from Mac Snow Leopard

Posted on by

Accessing my Ubuntu 9.04 Gnome desktop from the built in Mac OS X 10.6.2 VNC viewer took a bit of tweaking on the Ubuntu Gnome side. I have an OpenVPN SSL tunnel between the Mac and the Ubuntu desktop, however a SSH tunnel could also be used to protect the VNC session. In this post, I’ll just cover the VNC server setup assuming a secure connection between the Mac and the desktop.

Initially I followed the guidance at sanity, inc.”How to OS X Leopard Screen Sharing with Linux“, on Ubuntu I installed tightvnc:

apt-get install tightvncserver

Then tested it out by starting up the vnc server on the Ubuntu system as the user I want to run the remote session as:

tightvncserver -geometry 1024x700 -depth 24 :1

As tightvncserver starts up the VNC service, it will check for a .vncpasswd file in the user home directory. If it doesn’t exist, you will be prompted for a password to use to protect the remote session.  Note VNC is not designed to be used for multi-user remote access.
On the Mac, rather than use Bonjour to automatically discover the Ubuntu screen sharing service, I just referred to the VNC session directly within Finder which invokes the built in VNC viewer. Enter the VNC session password when prompted and the Ubuntu desktop is displayed. connect-to-server Within Finder, either use Go -> Connect to Server or Apple-K to bring up the Connect to Server window.  The server address is the URL that points to the Ubuntu VNC instance vnc://10.10.1.2:5901 where the port is 5900 + the display number specified when starting up the tightvncserver (5901).

This all worked fantastic, except for the keyboard mapping within Gnome – it was scrambled.  After googling several possible solutions, the only one that was successful for me was to disable the keyboard plugin in Gnome

Amit Gurdasani wrote on 2008-04-28: #51

I’ve also encountered this issue with TightVNC and the hardy release. My solution was to capture the xmodmap -pke output as ${HOME}/.Xmodmap at the login screen (DISPLAY=:0 XAUTHORITY=/var/lib/gdm/:0.Xauth sudo xmodmap -pke > ${HOME}/.Xmodmap). When gnome-settings-daemon starts up and finds an .Xmodmap, it asks if it should be loaded — I answer yes. As a side effect, if gnome-settings-daemon were to be restarted without the .Xmodmap, it’d scramble the keyboard layout again. With an .Xmodmap in place, it’ll load the .Xmodmap every time.

Due to another issue (#199245, gnome-settings-daemon crashing with BadWindow every time a window is mapped), I disabled the keyboard plugin using gconf-editor, at /apps/gnome_settings_daemon/plugins/keyboard. Since it’s not being loaded, I suspect it might not garble the layout even if I remove the .Xmodmap now.

So maybe disabling the keyboard plugin is a better fix.

On the Ubuntu system, invoke the Gnome configuration editor (gconf-editor on command line), then navigate to apps -> gnome_settings_daemon -> plugins -> keyboard uncheck the Active keyword.  Kill the VNC daemon and relaunch it – problem fixed.

pkill vnc
tightvncserver -geometry 1024x700 -depth 24 :1

Various methods exist to automatically start and kill the VNC server, but for now this will do it for me.

IMAP mailstore migration .. again

Posted on by

So last weekend, I discovered that Spamhaus decided it would be a good idea to place all of the public IP addresses for Slicehost (my Linux VPS hoster) into their Spamhaus block list (SBL). This covered both my slice in Dallas and the one in St. Louis – meaning an impressive chunk of inbound mail to my domains was being trashed by the sending MTA and an even bigger chunk of my outbound mail was being outright rejected since the sending IP’s were on the SBL.  Slicehost worked hard to convince Spamhaus to recind the blocklist, so the Slicehost IP’s got moved over to the less-nasty-but-you’re-still-probably-a-spamming-dirtbag Policy Block list (PBL) assuming affected IP owners would request to be removed from that list.

Sample query to see if you’re on any Spamhaus block list:  http://www.spamhaus.org/query/bl?ip=10.11.12.13

It seems it’s time to relinquish the care and feeding of my own Postfix mail system and turn to a hosted solution.  This means I need to migrate about 5GB of IMAP store to another site (again).  Last time I did a wholesale migration, I used imapsync to make the transition painless. In the code example below, an SSL connection to the IMAPS server at imap-server.sourcedomain.com is made with username@sourcedomain.com and the password stored in the plaintext file secret1. An SSL connection is made to the target system (which happens to be the server on which the imapsync tool is running, but could just as easily be another IMAPS server somewhere on a network accessible to the host where imapsync is running). The –delete and –expunge1 arguments will clean the successfully moved messages from IMAP store #1 .. so be sure you have your messages on the target successfully! Imapsync can be run iteratively to ensure you have got all the messages from your source.


/usr/bin/imapsync \
--host1 imap-server.sourcedomain.com \
--ssl1 \
--authmech1 LOGIN \
--user1 username@sourcedomain.com --passfile1 secret1 \
--host2 127.0.0.1 --user2 username@targetdomain.com --passfile2 secret2 \
--ssl2 \
--delete --expunge1 \
--buffersize=128

And one can use the

--dry

option to just test the process but not actually move any of the messages.

So that’s it – I’m about half way though migrating my current IMAP stores over to a hosted mail solution, so that I don’t need to keep up with the increasing level of care and feeding that running your own mail service requires.  Before I get too many darts about that .. I first started running my own personal MTA in 1995, adding spam and av filtering over time, and adding substantial redundancy (servers, sites, storage) so I could rely on it and fix things that broke as I had time rather than right when they broke (which was always at a bad time).  My new hosted solution takes over from two VPS servers running Postfix, Spamassassin, ClamAV, Greylisting with the IMAP store replicated across data centers in different states (15 minute rsyncs).  So soon, the (hopefully) last Allen Pomeroy owned and operated MTA can be turned off, while I get to work on fun stuff, rather than figuring out why my email is bouncing.  🙂

Update 2012/12/17:

Sometimes manual manipulation of your mailstore via IMAP is needed, so here’s how I deleted a large number of folders I had trashed and were being synced to my new system from the old.  Kinda clunky, since I didn’t get the scripted version to work (just used a copy/paste in an interactive bash session), but got the job done for now.

Connect to the IMAP server using SSL:
openssl s_client -crlf -connect imap.emailsrvr.com:993

* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Server ready director6.mail.ord1a.rsapps.net

Log in with your email credentials:
0 login user@domain.com Password

0 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS QUOTA] Logged in

List the folders you want to remove:
0 list "" "Trash.*"

That didn’t return the list I was expecting, so I listed all folders
0 list "" "*"

… and realized the source mail system adds “INBOX” on the front of the folder names, so then this command worked to list the folders to be deleted:
0 list "" "INBOX.Trash.*"

I copied the output and edited it to insert the folder name into a delete command:
0 delete "INBOX.Trash.Folder1"
0 delete "INBOX.Trash.Folder2"
0 delete "INBOX.Trash.Folder3"

0 OK Delete completed.
0 OK Delete completed.
0 OK Delete completed.

Finish off the session by logging out:
0 logout

* BYE Logging out
0 OK Logout completed.
closed

FreeMind mind mapping tool

Posted on by

Have you ever had a daunting task that just seemed like a nightmare to get your head around how to organize it? If you’re like me, you try to find some patterns in all the individual elements that make up whatever the topic is you’re trying to get a handle on. The patterns may not come easily, and even if they do, it’s usually a pain to try and re-categorize an element as you see fit (ever tried to create lists and categorize things in Excel??).

I came across a tool that one of my clients uses called FreeMind – it’s a Java app that allows you to enter a number of text elements and reorganize them in a hierarchical fashion.

FreeMind example

FreeMind example

Ok, one can do that with an unstructured word processor document or a spreadsheet, but FreeMind allows you to dump all these random ideas onto the page then drag and drop into categories or tags that make sense as you’re rearranging the elements.

So after about an hour of dropping in ideas around areas of improvement for the IT security of one of my clients, I had over 250 elements organized into 8 high level categories and about 18 subcategories. It was grouped well enough to lead discussions on what the current priorities for their programmes should be. If I had attempted this in a spreadsheet (and I had) it would have taken hours and untold frustration – not to mention I probably would have missed relationships that I could see in FreeMind.

If I had attempted this in a spreadsheet (and I had) it would have taken hours and untold frustration

FreeMind icons

FreeMind icons

You can add icons to each element to make labeling and categorization easier. Best to check out the FreeMind home page as it is a feature rich tool. From the project Wiki, typical uses include:

  • Keeping track of projects, including subtasks, state of subtasks and time recording
  • Project workplace, including links to necessary files, executables, source of information and of course information
  • Workplace for internet research using Google and other sources
  • Keeping a collection of small or middle sized notes with links on some area which expands as needed. Such a collection of notes is sometimes called knowledge base.
  • Essay writing and brainstorming, using colors to show which essays are open, completed, not yet started etc, using size of nodes to indicate size of essays. I don’t have one map for one essay, I have one map for all essays. I move parts of some essays to other when it seems appropriate.
  • Keeping a small database of something with structure that is either very dynamic or not known in advance. The main disadvantage of such approach when compared to traditional database applications are poor query possibilities, but I use it that way anyway – contacts, recipes, medical records etc. You learn about the structure from the additional data items you enter. For example, different medical records use different structure and you do not have to analyze all the possible structures before you enter the first medical record.
  • Commented internet favorites or bookmarks, with colors and fonts having the meaning you want

What a great tool .. I’m sure I’ll find more uses for it!