Archive for the ‘downtime’ Category

Expect some Downtime on a few Perceptus Websites

Sunday, November 9th, 2014

We’re doing some house cleaning on our web servers. Some of our websites have been, and will be, moving around over the next couple weeks. If you happen to get an error about the server being down, this would be the reason. For our more important websites, we will be taking great care to minimize the downtime.

This blog moved. No moving trucks required.

Friday, June 24th, 2011

The Perceptus Back Story blog was down for an hour or so yesterday while we migrated it from one virtual private server to another.

The reason for the move? It’s about time we install a few WordPress plugins that we’ve had our eye on.  We know that commenting on this blog is awkward – you have to register.  We’ll be playing around with 3rd party logins for commenting.  It might even happen today.

Plus, the old server is destined to be rebuilt sometime in the hazy future – the software on it is nearing it’s end of life.

Whoops. The Perceptus Forums needed some fixing.

Sunday, August 8th, 2010

Whoops. The login pages to the Perceptus Forums, has been broken, probably for a while – it’s a quiet forum, it was easy to miss.

Anyway, it has been fixed. It’s too bad we don’t get frequent posts, or else we would have noticed earlier.

Spoolsv.exe using lots of CPU on a Terminal Server

Friday, July 17th, 2009

I have this recurring problem at one client’s site.  The setup is a small Windows 2003 Terminal Server environment, with remote POS terminals over the Internet – I don’t like this arrangement for the many single points of failure (a couple ADSL lines, a couple routers, a couple switches, a single server, etc.), but that’s beyond the scope of this post.

This client is happens to be running BBL, AKA WinPOS, AKA, WinBPS, over Terminal Server.  I have no comment on the software itself, it’s the arrangement of using it over Terminal Server over the Internet that makes me nervous – no, I had no input into this setup.

A further complication, is that the printers that the software uses, Zebra, label/ticket printers, have poor drivers that I don’t believe to be TS compatible.  They sometimes get stuck when the remote session is closed.  This eventually screws up the Windows spooler service.

When this system stops working, new Remote Desktop sessions fail to start properly.  After logging into the server, I will check the processor usage in Task Manager.  If the spoolsvc.exe process is using a lot of processor time, this is what I do:

N.B. Follow at Your Own Risk.

Stop the spooler service (no one will be able to print, but they probably can’t already):

net stop spooler

Open RegEdit.exe and browse to this part of the registry:

HKLM\SYSTEM\CURRENTCONTROLSET\CONTROL\PRINT\PRINTERS\

At times, I’ll find dozens of stale Terminal Server printers listed.

Export the registry keys first, to make a backup, just in case.

Then delete the Terminal Server printers from that same part of the registery, do NOT delete the local printers.  These are the ones with the “…on MyComputer … in session…” within the printer name.

Deleting these has worked for this particular server several times to resolve the CPU hogging spooler service (spoolsv.exe)… but sometimes they can’t be deleted in RegEdit.

Grab a copy of RegDelNull by Systernals, it’s free as in beer.

Use RegDelNull.exe to remove some of the entries that would not delete manually – I’m looking at you Mr. Zebra Printer Driver.

Here’s what that looks like (anonymized, of course):

C:\Installs>regdelnull HKLM\System\CurrentControlSet\Control\Print\Printers -s

Note: In a recurrence of this problem, I had to run RegDelNull one step higher in the registry, I’m not sure why – i.e. HKLM\System\CurrentControlSet\Control\Print

RegDelNull v1.10 - Delete Registry keys with embedded Nulls Copyright (C) 2005 Mark Russinovich Sysinternals - www.sysinternals.com Null-embedded key (Nulls are replaced by '*'): HKLM\SYSTEM\CURRENTCONTROLSET\CONTROL\PRINT\PRINTERS\Zebra101 _ on MyComputer(from MyComputer2) in session 3\PrinterDriverData\Barcodes\(0x1701)*de 93

You get a prompt to fix the Null.  Say “Yes”.

Then go back up a few steps and delete the problematic printer from RegEdit.  With the null value gone, you can delete it.

Then restart the spooler service.

net start spooler

If this works, the spooler will start, it won’t use a suspicious amount of processor cycles, and the RDP connections will start working immediately – no reboots required.

Then go and find updated drivers.

I’d be glad to hear about a better solution… please comment below.

Reducing Memory Used by Milter-Greylist

Monday, July 21st, 2008

Our VPS was running low on free memory the last few weeks.  After a bit of research, we realized that our email greylisting software, Milter-Greylist was using the most memory of everything installed on our server.  More than our database engine, web server, email server, and everything else (not combined)!

For those who don’t know, Grey Listing delays emails in an attempt to foil spammers which don’t typically follow standards for retrying email messages. Milter-greylist is a package that works with sendmail, our SMTP server. Milter-greylist is great, however, it keeps it’s working history in memory, which was OK for the two years that we have run it.  However, the amount of spam attempts continues to rise… why don’t home users notice that their computers have become SPAM zombies anyway?

So, the milter-greylist was storing tens of thousands of records in memory.  It had to be reduced.  Rather than switch to a database driven greylisting package, we decided to start blocking some SPAM attempts earlier in the process.

We enabled the outright blocking of inbound email attempts by any IP address listed on Spamhaus.org’s SBL+XBL list.  SBL+XBL are lists of computers (built by crazy wizardry) that one can use to blacklist email attempts.  I’m uncomfortable using blacklists like this, but, what can you do?  The odds of good mail being lost are very small, and hopefully, anyone who happens to get bounced unintentionally can phone us.

So, following the simple instructions here:

http://www.joeldare.com/papers/spamhaus.pdf

We were able to reduce the traffic to Milter-Greylist and it’s memory usage by 2/3 to 3/4!  Uptime and performance of our VPS and therefore everything hosted on it should be slightly better.

Spammers suck.