OK, it’s time.  I’ve waited long enough before delving into this topic.  After helping someone access my blog today by allowing invalid HTTP headers that their firewall was sending, I figured it was time to deal with this.  So let’s talk about internet security.  (And since I’ve worked with the internet for decades—since long before “web” was even a twinkle in technology’s eye and since Windows was a paltry version 1.0, let’s assume I’m smarter and more informed about these things than you are.  I’ll happily debate that point with anyone, by the way.)

(1) Everyone should be running antivirus and firewall software.  And everyone should be keeping that software up to date.  Period.  If you’re not running both and if both are not up to date, get off the internet now and don’t come back until you remedy the situation.  This is not negotiable because you pose a threat to yourself and everyone else on the internet.

Spammers and hackers use internet proxy servers so they can hide behind someone else’s IP address.  Do you know where these proxy servers come from?  Some are real servers provided for that purpose, some are servers which have been hacked, and the rest are individual computers and smartphones and other devices which have been infected by malicious software.  Most fall in the last category—user devices, and that means that most people aren’t running antivirus and firewall software, and of those who are, many are not keeping that software updated.  Shame on you!

(2) Local firewalls should not bastardize HTTP headers.  If your firewall says it will block the “referer” field, then it should remove that data from the HTTP header or it should send a blank field, but it should not replace it with munged entries.  Some replace it with all hyphens and some replace it with misspellings (e.g.; “Weferer”).  Neither of these is a valid HTTP header, and therefore both can legitimately be blocked with a 400 response (bad request).

You see, any header your browser sends to a web server has to be usable if the server wants/needs it.  That’s what the headers are for when your browser sends them.  By definition, they must be usable by the web server.  And that means sending me a header labeled “——” is a bad request.  Worse yet is the software that does the same thing to “Content-Encoding” and other fields, which strikes me as ignorance beyond measure, yet the hyphenated “referer” field is created by the same software that also hyphenates the “content-encoding” field.  Dumb!  And invalid!  (And the source of today’s problem.)

Asking a web server to accept such HTTP headers is asking that web server to open itself up to nefarious individuals.  If your software wants to send invalid HTTP headers to a web server, then it’s software written by and for hackers, spammers and other bad people.

(3) You have every right to be anonymous on the internet so long as you can be identified by legal means should the need arise.  But your browser has no right to be anonymous.  There’s an application for Mac computers that turns the browser “User-Agent” HTTP header into something like “Mozilla/5.0 (000000; 000000000000; 00000000) 0000000” or whatever.  This is pure stupidity!

Web servers perform content negotiation based on browser capabilities and versions.  For example, some versions of Microsoft’s Internet Explorer report that they understand compressed encoding of the HTTP stream when in fact they don’t, so it’s up to the web server to check the browser’s make and model so the server can determine if it should compress the stream or send it as is, and all regardless of what the browser reports.  But if the browser hides its identity, it gets binary crap in response to a GET request or it gets a huge chunk of uncompressed data instead, all because the server can’t tell if the request comes from a source capable of handling compressed data.

Also, many web servers now offer content presentation based on platform and browser, like giving a streamlined version of the site for smartphones and a full version to regular browsers.  But if your computer is reporting an invalid browser, the server can’t determine the optimal format for the data and so has to send the default.  Which means you’re getting a suboptimal presentation for your platform.

So turn off any obfuscation of browser identity because it’s stupid, it’s invalid, and it ensures you get a web page fit for the lowest common denominator of browser—assuming you aren’t just blocked for sending such riffraff to a web server that knows better.

(4) I see people on Facebook bitching and moaning all the time about how Facebook has been hacked.  Liars!  Facebook wasn’t hacked; your account has been hacked, either because you have a weak password or because you clicked on a link and gave some arbitrary application access to your account.

People need to take responsibility for their own actions, including the tendency by most to click on whatever link they see and pressing “OK” on any prompt they encounter.  This is abhorrent behavior since many social networks grant access not only to your data, but also to the data of all of your connections (e.g., “friends”).  So stop clicking things arbitrarily and stop saying “Yes” every single time something says it wants access to your account.  You’re a menace!  You endanger my privacy by your inability to control yourself.

(5) Disable JavaScript and Flash on all web sites by default.  You can enable these functions for those web sites you trust.  Leaving these methods open is a very bad idea.

Using JavaScript and a major security hole in CSS, I can write a web page that will easily walk through the web sites you’ve visited, thereby determining what sites you visit and what sites you don’t visit, and then I know all about your surfing habits.  Using JavaScript, I can call on any plugins you’re running, I can talk to the browser at a level you can’t see in a page, and I can access data that would shock you.

Adobe’s Flash player is one of the most horrific security problems to hit the internet (assuming that one accepts Internet Explorer as the biggest security problem).  Flash has never had a stable, secure version.  In fact, it continually shows up on security alerts as having yet another set of problems.  And unfortunately, Flash doesn’t provide easy access to its most meaningful security settings; they can only be accessed by visiting the Adobe site.  That means 99.999% of people have never changed those settings.  So in addition to Flash’s continuing security flaws, it also harbors a by-design security hole created by limiting options access to specially coded pages that are never made obvious to most users.

So disable both.  Turn them on only for sites you trust.  And following that, any site that says it can’t show you anything unless you enable one or both is a site you shouldn’t visit, because it’s a site with ignorant programmers or a site with ill intentions.

(6) Keep your browser up to date.  People still running Internet Explorer 6 or Firefox 3 or Opera 9 deserve no sympathy.  Browsers are updated for reasons, not the least of which is to address major security issues.  Sure, you get better performance and more awesome functionality, but really it’s about plugging the holes that bad people use to enter your life.

So update your browser.  Regularly.  No matter what platform you use.

(7) If you’re a server administrator, you need to get your shit together.  Make sure you’re running a firewall, make sure you’re running log monitoring (to check for nefarious activity), and make sure you’re regularly checking for bad things on your boxes.  There’s nobody to blame except yourself if your site/server is being used by bad people to do bad things.

— — — — — — — — — —

Notes:

1. I have seen a 99.9999% reduction in spam since enforcing HTTP standards and server rules (i.e., not letting servers connect while they claim to be users).  The only spam I’ve seen in the last nine months has been from real people browsing and submitting spam.  That says something, especially because my frame of reference is across multiple servers and dozens of sites.
2. I intend to begin releasing my anti-spammer/anti-hacker technology information soon.  This will include .htaccess/modsec rules along with the supporting data.
3. If the do-good technology makes it inconvenient or impossible for you to be comfortable, that’s your problem and not the problem of a secure webmaster.  Sending me invalid HTTP headers does not make you a victim…
4. I have no sympathy for anyone running Internet Exploder—er, uh, I mean Internet Explorer.  It’s the worst browser ever created—IN THE HISTORY OF THE INTERNET!—and it’s the most common vector for internet-based malware infection.  Upgrade to a real browser, one that’s standards compliant and secure.
5. If you use a Mac or Linux, don’t feel smug.  Though Windows has you beat hands-down with regards to insecurity, that’s only because it has you beat hands-down with regards to numbers: Microsoft simply has more users than you do.  Your issues are just as relevant as theirs.