Windows Updates to go… not so much?

I posted a few weeks ago about bringing a WSUS machine down to Haiti as a sort of proxy – unfortunately, this project turned out to be a bust.

Conceptually, it worked. Machines were getting updates, and everything was great… until it started blowing out the CMOS settings and bluescreening a couple times a day. I suspect it’s probably power-related, but in any case, it was far from stable enough to leave unattended.

Windows Updates, To Go!

When I leave for my trip to Haiti in a few weeks, one of the things I’ll be doing is bringing multiple computers up to current patches. There are a few ways to do that:

One is to bring some sort of removable media (optical or flash stick) down and apply them manually. The problem with this is that once I leave, the machines stay in their current state until the next geek can come down and apply the next batch of patches. Downloading patches for multiple machines over developing-world internet connections can easily run into daily bandwidth caps, and Windows Update doesn’t cache very well through a normal proxy server such as Squid.

Another is to use Windows Server Update Services (WSUS). I initially considered setting up a Windows Server VM on my laptop, syncing up the updates stateside and temporarily configuring the machines down there to pull from my impromptu update server. Then I got the idea that a lightweight appliance-type server that lived down there permanently would be a useful solution that would download the patches once and distribute them over the LAN. Since we’re planning on using Microsoft Security Essentials for anti-malware, this solves the problem of definition updates. Daily patch sync would happen in the wee hours of the morning when the oversubscribed connections in Haiti are generally pretty clear.

I rummaged around the office and found a Dell FX160 thin client that we got as a demo unit from Dell (I have a number of blog posts on the topic of this device). It has been gathering dust for some time as it’s hobbled with a 1GB SATA flash disk and limited RAM. After checking on hardware requirements for both Windows Server and WSUS, I went out and picked up a 120GB SSD and a pair of 2GB RAM sticks and put them in. The choice of an SSD wasn’t so much for performance reasons (although it can’t hurt), but for the machine to be entirely solid-state. It’s going to live in a fairly harsh environment where mechanical failures are likely.

Once I got the hardware put together, I hooked up a USB optical drive and loaded Windows Server 2003 R2, and then installed WSUS and performed an update sync. The whole process went mostly smoothly.

Here are a few of the gotchas in installing Windows 2003 on an FX160 thin client, a job it was NEVER meant to do:

  • SATA controller needs to be in ATA mode. If it’s in AHCI mode, Windows 2003 will not recognize the disk.
  • When using a storage device that the BIOS recognizes as a hard drive, it expects to see a fan plugged into the motherboard. This fan is part of the hard drive bracket kit (Dell P/N H224H). When a fan is not detected, each boot will require a manual intervention during POST to press F1.
  • Stock Windows 2003 media does not include video drivers or network drivers for the FX160 (Broadcom NetXTreme 57XX).
  • Dell’s support site doesn’t have the most recent drivers for the Broadcom.
  • It’s virtually impossible to find a 6″ SATA extension connector, either for data, power, or both. I was finally able to find a power extension, but used a standard SATA cable to connect to the other SATA port on the motherboard.

The SSD I used for this is an OCZ Agility 3, 120GB. Disk performance on large writes is almost 100MB/sec, which is about twice as fast as my 7200RPM spindle drive in my laptop. Windows performs very well with 4GB, a SSD, and a 1.6GHz Atom processor.

The next step was to configure the clients to update from the server for testing. I still have one of the Asus netbooks that we deployed to Haiti in a previous trip. This is where I discovered that Windows Home and Windows Starter don’t include the policy editor (gpedit.msc) that I’m used to finding on Pro/Enterprise/Ultimate versions of windows. This is understandable, your average home user doesn’t (and shouldn’t) normally jack with system policy. Fortunately, all the policy editor does is manipulate registry keys, and the process of configuring Windows Update via the registry is well documented. This actually simplifies things, since all I have to do is create a .reg file that I can import on all the target machines.

Next post: Installing Squid. Not content to use this box for mere update caching, we’re gonna have it be our web proxy as well.

Configuring Perl for ASSP on Debian

Quick and dirty apt-get string to install all the requisite perl modules (and associated dependencies) for ASSP on Debian:

apt-get install libcompress-zlib-perl libdigest-md5-file-perl libdigest-sha1-perl libemail-valid-perl libemail-send-perl libemail-mime-perl libfile-readbackwards-perl libclamav-client-perl libweb-simple-perl libmail-spf-perl libmail-srs-perl libnet-cidr-lite-perl libnet-dns-perl libnet-ldap-perl libnet-smtp-server-perl libunix-syslog-perl

Veeam’s Next Big Thing

VeeamHyper-VIt’s official – Veeam is announcing this morning that version 6 of their award-winning backup/replication software will support Microsoft’s Hyper-V virtualization hypervisor. The new version is due out later this year.

What’s Cool about Veeam and Hyper-V

Veeam is once again delivering IT magic by building their own Changed Block Tracking functionality into Hyper-V for some of the highly efficient backup and replication that Veeam is known for. This is going to go a long way toward bringing Microsoft virtualization up to par with VMware. Also included are file-level restore and virtual lab provisioning, as well as SCOM integration.

For non-profits, this is potentially huge, since it brings advanced backup capabilities to the hypervisor that’s included with Windows Server. VMWare is great technology, but for SMB and non-profits, VMWare’s pricing point is painful. When non-profit/education customers can get Windows Datacenter licenses for around $300 a socket (which includes Hyper-V!), suddenly VMWare looks really painful, even after educational discount.

What’s still missing

The initial release will not include Veeam’s U-AIR capability, but they’re hard at work to bring that capability online.

It also lacks the ability to back up/replicate across virtualization platforms, but that’s to be expected.

Veeam hasn’t yet announced pricing/licensing details. What I’d really like to see from Veeam is a per-socket license that is platform-independent.

If you haven’t yet experienced the awesomeness that is Veeam, give the folks at Mirazon a call. Those guys know Veeam up and down and backwards.

Veeam’s Rick Vanover has more at his blog.

Kicking Skype Up a Notch

A few weeks ago, our Senior Pastor asked for some assistance with setting up a skype video conference so that Adam could participate in a meeting being held in Texas. The alternative was to have him fly down to Dallas for a 1-hour meeting, effectively blowing out an entire day of productive hours.

We don’t currently have a dedicated video conference system, so we had to improvise.

We scheduled the meeting in our studio and coordinated with the other end to make the conference happen via Skype.

On our end, we took Adam’s MacBook Pro, and hooked up a Canon XL2 via FireWire for the video, a Shure wired lapel mic hooked to the camera (for phantom power) and then the audio output from the camera into the MacBook’s line-level audio input (because it appears that Skype doesn’t recognize the audio device on the XL2). We then connected the audio and display output from the mac into a 40″ LCD TV.

Here’s what it looked like:

The end result is a conference that looks and sounds excellent.

Fixing network Priority in Windows : Win7 Update

A long time ago, I made a post about fixing network priority in Windows, and I found myself having to do the same task again on my new Windows 7 system. The process isn’t quite as easy to find under Windows 7/Vista. Here’s the updated version:

Right-click on your network icon and go to the “Network and Sharing center” (if the “Network” icon is on your desktop, you can also get there by right-clicking and going to properties)

Click on “Change Adapter Settings”

Network Advanced

Press the “Alt” Key to show the menu, and click on “Advanced”, then “Advanced Settings”.

(from here, the process is unchanged)

Move the Wired LAN Connection (By Default, “Local Area Connection”) to the top, followed by the wireless connection. Make sure that any VPN virtual adapters come after these, otherwise the VPN will only use the ones above it. This tends to be problematic if you’re using split tunneling, as it will kill any network connection you have.

Once you’ve applied the settings, open a command prompt and run “nslookup” – it should default to the DNS server for your wired network.

Say Whaaaaaaaa?????

The last few days have been rather stressful.. Our shiny new web infrastructure at COR has been throwing major temper tantrums, which means I’ve been rather busy of late. Today, it melted down half a dozen times after I thought I’d fixed it. Each time it did something different. (and if that wasn’t enough excitement, our upstream provider had a BGP issue this morning that knocked their entire customer base off the web for about 5 minutes)

All you folks that hit our website, thank you for your patience. These have been trying times.

This morning, I noticed something very odd. And again this evening as I’m migrating the data to a new server.

root@corweb1:/content/sites# uptime
11:35:56 up -24855 days, -3:-14,  4 users,  load average: 0.00, 0.00, 0.00

Negative uptime??? What?

root@corweb1:/content/sites# date
Wed Aug 31 11:50:17 CDT 1955

Perhaps running apt-get install flux-capacitor wasn’t such a good idea, as the machine seems to be performing on about the level of a computer from 1955.

It’s become pretty obvious that something is very ill on that box, and I think it’s time to ditch VMWare Server for ESXi. Until then, we’re moving the servers over to the bare metal on the other box (which includes the blog server, it’s already been moved).

Last login: Wed Aug 31 11:44:51 1955 from

No wonder I feel old.

Turn the radio on!

(apologies to Randy Travis for lifting a title)

On Friday, our vendor came out to replace the radio on the Southcreek end of our wireless link. (More on that at Clif’s Blog). Long story short, we improved the income side of the link budget by about 16dB.

Got this done just in time for a big rainstorm on Saturday, followed by sloppy wet driving snow on Sunday (attendance was way down, partly due to the weather. Some churches even canceled service. Well, sort of.) Even Kansas City International Airport had its longest closure in history because they couldn’t keep the runways clear long enough. We Canadians are amused by this notion.

Since we had just gotten a shiny new radio and antenna on the Southcreek end, I was curious to see how the link was performing in the snow. I fired up WhatsUp and checked my wireless status page. Both bridges showed more or less the same thing:

(Time of day is along the X-axis, and the Y-axis is received signal level (RSL) in hundredths of a dB, so -3100 is -31dB – due to a firmware update, it only reports in whole dB now, probably because the fractional numbers weren’t nearly as accurate as they were precise )

The pattern struck me as intriguing, because precipitation generally looks a little different, as demonstrated by Saturday’s rainstorm (you can also see the beginning of the snow on the far right):

After checking a few weather sites, I discovered that the downward slope at 6:00 correlated to the beginning of the snow. I was beginning to suspect that at least one of the radomes was plastered in snow. We’d just gotten back from church, where the wind was blowing pretty hard from the northwest, and the Central Campus end was facing almost directly into the wind at the top of the building. I asked my wife if I could run back and do a little weekend science. After realizing that this sort of thing was part of what she signed up for when she married a geek, she sent me on my way with the camera (thank you honey, I love you! *smooch*)

I stopped by the Southcreek office first, and realized that the blue Bridgewave logo on the radomes was going to be very helpful at determining accumulation. This is what Southcreek looked like:

(apologies for the grainy picture, it was taken from about 100 feet away at max digital zoom and then cropped):

Unsurprisingly, there was no significant accumulation on the Southcreek radio, as the radome was facing downwind. This is what the weather looked like towards the other end of the link:

I drove over to the church, where the conditions looked like this:

Notice that the snow is plastered on one side of the trees. The CC radio is facing that direction.

I found a radio and got a hold of George (on the facilities team, also does desktop support for us one day a week) to let me onto the roof. George looked at me funny and wondered why I wanted up on the roof in this craptacular weather. After a brief explanation, he joined me (and wanted to see for himself, too – George is a geek at heart). I get up on the roof, and do a little skating (roofing membrane is nice and slick when wet, never mind when covered in a few inches of sloppy wet snow!)

Sure enough, here’s what the radome looked like:

It was pretty clear what was causing our 30dB signal loss (the link was still up, with about 10dB to go). George went off to find something to clean off the snow (it’s about 15′ from where we were standing, and we didn’t have a ladder). While George was off playing MacGyver, I got to thinking that the snow probably wasn’t stuck on very well, and that some sort of jarring impact might knock it off. If only I had something to throw at it… Like, say, a snowball. My concern was that the snowball would stick to the radome and REALLY attenuate the signal, but I figured this stuff was wet and slushy enough to form into a ball, but was too wet to actually stick to anything (it was above freezing the whole time). So I started chucking snowballs at a piece of gear that costs about the same as a decent new car (I love my job!). On the third try, I made solid contact just below the logo, and the sheet of snow came sliding right off (look below the right loop of the logo for the point of impact):

(by the time I actually got the picture taken, some more snow had accumulated on the radome. Did I mention it was snowing hard?)

I went down to a computer to check on the signal level. Sure enough, the link improved a bunch. (I’ll repost the image here so you don’t have to scroll all the way to the start of the post.) The snowball caused the sharp vertical spike on the right side of the graph. The picture was taken about the spot where it dropped back down a few DB:

I headed back for the roof and found George had MacGyvered a pole from an extendable dusting wand and a wooden broom handle, held together with packing tape. I climbed back up onto the roof and was able to reach the radome with George MacGyver’s snow brush. Cleaning it off gained me a few more dB (second, smaller vertical spike on the graph):

As you can see on the graph, some more snow started accumulating, and then the snow stopped and started melting off. By mid-afternoon, the sun had come out we were back up to our normal signal levels, and there was little evidence left around town that we’d even had a snowstorm. We went from this, where it’s snowing sideways…

…to a beautiful sunny day in a matter of hours. I’m glad I didn’t bother shoveling my driveway, as it had melted clear by the time my wife and I got back from the movies (we went to see Jumper. Good flick, but left a lot of unanswered questions — sequel, anyone? — as well as leaving me with lingering nausea from the jumpy camera work)

I haven’t heard what the attendance was like at the 5:00 service. Morning services were sparse due to weather, but Rev. Junius Dotson from Saint Mark UMC in Wichita was our guest preacher this week and preached a great sermon (Adam is off in Colorado enjoying the real snow with the high schoolers). I hope a bunch of folks got to experience Rev. Dotson at the evening service. The man just has style.

And now, for the ADD folks that lost me about 6 paragraphs ago, here’s a nice little summary:

Following the spam…

Mark Wade at CA recently posted a neat article about what happens when you respond to a spam message. Check out some of Mark’s other posts as well. Lots of good stuff there for our users, but in a forum where your typical end-user is not likely to be found.

I will occasionally chase down phishing messages (and populate with bogus data) just to see how elaborate the phish is, so I can warn our users. There have been a few recently that have been particularly well crafted, and thus pose a higher threat, as the deception is far more effective.

Just a reminder for your users, unless your bank is run by a bunch of idiots, they won’t e-mail you out of the blue. If they do, it’s time to change banks.