The Perceptus Back Story

I have this recurring problem at one client’s site.  The setup is a small Windows 2003 Terminal Server environment, with remote POS terminals over the Internet – I don’t like this arrangement for the many single points of failure (a couple ADSL lines, a couple routers, a couple switches, a single server, etc.), but that’s beyond the scope of this post.

This client is happens to be running BBL, AKA WinPOS, AKA, WinBPS, over Terminal Server.  I have no comment on the software itself, it’s the arrangement of using it over Terminal Server over the Internet that makes me nervous – no, I had no input into this setup.

A further complication, is that the printers that the software uses, Zebra, label/ticket printers, have poor drivers that I don’t believe to be TS compatible.  They sometimes get stuck when the remote session is closed.  This eventually screws up the Windows spooler service.

When this system stops working, new Remote Desktop sessions fail to start properly.  After logging into the server, I will check the processor usage in Task Manager.  If the spoolsvc.exe process is using a lot of processor time, this is what I do:

N.B. Follow at Your Own Risk.

Stop the spooler service (no one will be able to print, but they probably can’t already):

net stop spooler

Open RegEdit.exe and browse to this part of the registry:

HKLM\SYSTEM\CURRENTCONTROLSET\CONTROL\PRINT\PRINTERS\

At times, I’ll find dozens of stale Terminal Server printers listed.

Export the registry keys first, to make a backup, just in case.

Then delete the Terminal Server printers from that same part of the registery, do NOT delete the local printers.  These are the ones with the “…on MyComputer … in session…” within the printer name.

Deleting these has worked for this particular server several times to resolve the CPU hogging spooler service (spoolsv.exe)… but sometimes they can’t be deleted in RegEdit.

Grab a copy of RegDelNull by Systernals, it’s free as in beer.

Use RegDelNull.exe to remove some of the entries that would not delete manually – I’m looking at you Mr. Zebra Printer Driver.

Here’s what that looks like (anonymized, of course):

C:\Installs>regdelnull HKLM\System\CurrentControlSet\Control\Print\Printers -s

RegDelNull v1.10 - Delete Registry keys with embedded Nulls
Copyright (C) 2005 Mark Russinovich
Sysinternals - www.sysinternals.com

Null-embedded key (Nulls are replaced by '*'):

HKLM\SYSTEM\CURRENTCONTROLSET\CONTROL\PRINT\PRINTERS\Zebra101 _
on MyComputer(from MyComputer2) in session 3\PrinterDriverData\Barcodes\(0x1701)*de 93

You get a prompt to fix the Null.  Say “Yes”.

Then go back up a few steps and delete the problematic printer from RegEdit.  With the null value gone, you can delete it.

Then restart the spooler service.

net start spooler

If this works, the spooler will start, it won’t use a suspicious amount of processor cycles, and the RDP connections will start working immediately – no reboots required.

Then go and find updated drivers.

I’d be glad to hear about a better solution… please comment below.

Our VPS was running low on free memory the last few weeks.  After a bit of research, we realized that our email greylisting software, Milter-Greylist was using the most memory of everything installed on our server.  More than our database engine, web server, email server, and everything else (not combined)!

For those who don’t know, Grey Listing delays emails in an attempt to foil spammers which don’t typically follow standards for retrying email messages. Milter-greylist is a package that works with sendmail, our SMTP server. Milter-greylist is great, however, it keeps it’s working history in memory, which was OK for the two years that we have run it.  However, the amount of spam attempts continues to rise… why don’t home users notice that their computers have become SPAM zombies anyway?

So, the milter-greylist was storing tens of thousands of records in memory.  It had to be reduced.  Rather than switch to a database driven greylisting package, we decided to start blocking some SPAM attempts earlier in the process.

We enabled the outright blocking of inbound email attempts by any IP address listed on Spamhaus.org’s SBL+XBL list.  SBL+XBL are lists of computers (built by crazy wizardry) that one can use to blacklist email attempts.  I’m uncomfortable using blacklists like this, but, what can you do?  The odds of good mail being lost are very small, and hopefully, anyone who happens to get bounced unintentionally can phone us.

So, following the simple instructions here:

http://www.joeldare.com/papers/spamhaus.pdf

We were able to reduce the traffic to Milter-Greylist and it’s memory usage by 2/3 to 3/4!  Uptime and performance of our VPS and therefore everything hosted on it should be slightly better.

Spammers suck.

Quick note to all who use the various Perceptus web sites, particularly our bingo card site and our hosted survey site:

Our virtual private server hardware was upgraded.  A couple hours of downtime occurred.  We’re sorry for the inconvenience.  Unfortunately, our provider did not give us much of a warning. Things seem to be up and running, but a bit slow – I think that will fix itself soon.