Jan Harasym

Designing highly scalable/resilient infrastructure by day; running hacker communities by night.

Page 2


Please stop advocating wildcard certificates.

Ever since I started “doing the computer stuff” I’ve been a bit wary of SSL/TLS.
It’s very easy to get wrong and revocation is not a solved problem no matter which security vendor is trying to push for sales.

There is, however, a strong urge to do things as easily as possible. For most people using SSL/TLS this becomes:

  • Use openssl commands from a 5 year old blog to generate a CSR.
  • Paste your CSR onto some web form.
  • Copy/Paste ciphers/config from some other website.
  • Ensure usage of wildcard certs. so you never have to do this pain again.

Some of these steps are worse than others, there’s almost no risk of using old openssl commands (except smaller keysize), but if you have an old list of cipher suites you’ll probably be using a deprecated cipher- and your clients data may be vulnerable in transit.

The worst one by far is the wildcard certificate; and this is for mostly obvious...

Continue reading →


Dropping filesystem caches.

By writing to /proc/sys/vm/drop_caches the linux kernel will drop clean caches, dentries and inodes from memory, causing that memory to become free. (yay!)

To free pagecache:

echo 1 > /proc/sys/vm/drop_caches

To free dentries and inodes:

echo 2 > /proc/sys/vm/drop_caches

To free pagecache, dentries and inodes:

echo 3 > /proc/sys/vm/drop_caches

As this is a non-destructive operation, and dirty objects are not freeable, the user should run “sync” first in order to make sure all cached objects are freed.

This tunable was added in 2.6.16.

It’s pretty handy when trying to benchmark disks.

Sidenote: on FreeBSD this is impossible, you have to unmount the disk and remount it.

View →


Moving away from GitHub.

 The tale of centralised source code management.

 Prologue

For a long time I’ve been a user of github, for personal public projects it has served me well enough and “github is the home for everything modern” has been sort of ingrained into my head for some time.

For private things we at darkscience and at libsecure had used Atlassian bitbucket, since they have a free tier that allows only private repos, and github has the same thing but about public repos, they complimented each other and our objectives perfectly. There was a minor hassle of course, but generally for a frugal group of people such as us, it was perfect.

 And then, the shitstorm;

ok, not a real shitstorm, more like “increasing amounts of discomfort”, github were beginning to censor and take down random repositories.

It’s not that we do anything particularly bad at darkscience but we generally err on the side of...

Continue reading →


How to write good.

  1. Avoid Alliteration. Always.
  2. Prepositions are not good words to end sentences with.
  3. Avoid cliches like the plague. They’re old hat
  4. Comparisons are as bad as cliches.
  5. Be more or less specific.
  6. Writes shouldn’t generalise.
    Seven. Be consistent.
  7. Don’t be redundant; don’t use more words than necessary; don’t be superfluous.
  8. Who needs rhetorical questions.
  9. Exaggeration is a billion times worse than an understatement.

View →


SaltStack notes

 Primitives

 Minions

Minions: salt “clients”, aka hosts / provision targets. (not to be confused with the salt command-line client salt)

 Master

master: the salt server, drives the provisioning of minions. the salt cli client runs on the master. The master is an ensemble of several services and worker processes.

  • Publisher (port 4505): which minions must be able to access for pull-mode
  • EventPublisher (IPC only):
  • MWorker: one or more “master workers”, which handle salt operations concurrently
  • ReqServer (port 4506): pop work and push to MWorker, plus receiving replies so MWorker doesn’t have to block
  • File Server (?): transfers files to minions on demand from the state tree

 Grains

Grains are basically facts in the ansible/puppet world.

 Pillar

Pillar is a global value/config storage, spelled out on the master. This is basically YAML which is laid out in folder hierarchies...

Continue reading →


Friends don’t let friends use BTRFS for OLTP

I usually write rant-style posts, and today is no exception. A few months ago I was working on a benchmark comparing how PostgreSQL performs on a variety of Linux/BSD filesystems, both traditional ones (EXT3, EXT4, XFS) and new ones (BTRFS, ZFS, F2FS, HAMMER). Sometimes the results came out a bit worse than I hoped for, but most of the time the filesystems behaved quite reasonably and predictably. The one exception is BTRFS …

Now, don’t get me wrong - I’m well aware that filesystem engineering is complex task and takes non-trivial amount of time, especially when the filesystem aims to integrate so much functionality as BTRFS (some would say way too much). Dave Chinner stated that it takes 8-10 years for a filesystem to mature, and I have no reason not to trust his words. I’m not a XFS/EXT4 zealot, I’m actually a huge fan of filesystem improvements (and I don’t really like EXT4 so much)...

Continue reading →


Theatre: Lolita

 (@ London Theatre)

I recently (as of 20 minutes ago actually) attended a production of Lolita, a representation of Stanley Kubricks work (they say on posters).

I had gone in with no expectations, well, when you purchase tickets for “The London Theatre” online you expect something grandiose in the heart of theatreland.

However, this was not one of those. This was a “Fringe Theatre”, which I’ve never heard of- but I’m open minded enough, although it’s situated in New Cross (not exactly known for it’s cultural prowess).

When we arrived at New Cross Gate station we were invited to walk over a rather sketchy looking scaffold bridge between platforms if we wanted to leave; once we got outside we navigated through the even sketchier neighbourhood

I’ve walked through New Cross before (back when I lived in Lewisham) and back then I had been hardened from my time in Coventry however, I’m a...

Continue reading →


The Sad State of British Broadband

I suppose I should change the title; it’s unfair to blame the broadband provider[0] and exclude the misdeeds of 3G/4G providers.

I have been at odds with the only true ADSL provider in the UK for some time. In fact, for as long as I’ve lived in the Capital.
I’ve lived in Lewisham (SE13), Aldgate (E1), and I’ve been living in Bow (E3) since July of last year and during this time I have achieved the average speed of 0.21Mb/s (yes, bits).

 4G Rollout

During this time 4G was rolled out across London, and despite not having signal in my home, I can in fact, use this new technology.

However this rollout has been delayed by almost 2 years, there was an auction for the 4g spectrum from ofcom[1], however, “EE” (formerly T-mobile and Orange) seem to have deployed nearly a year before anyone else. Whether that was ability or willingness I’ll never know. (my initial guess was that the company...

Continue reading →


Windows 7 Clients on Samba Domain

Today, I had to face the undocumented mess that is: adding a windows machine to our UNIX infrastructure.

 Why?

Where I work, we’re mostly UNIX and Linux, with UNIX on the backend for everything (solaris) and Linux for the e-commerce platform, along with the Customer Service computers. This is a stark contrast with people who are only accustomed to using Windows. Combine these factors (undocumented unix/windows + requirement to run windows) and the approaching April end of support deadline, and you have my heart racing and cold sweats.

I noticed that I can get some HP Prodesks (with windows 7 Pro) for less than the price of a Windows 7 license, so I bought one.

it was a modest machine with an AMD processor running 1.5Ghz and quad-core, but the improvements in CPU and harddisk design put it far above the other machines in the office for performance, which, shocked me somewhat.

As for...

Continue reading →


Failing to monitor, dying without dignity.

Today, I’m going to tell you about the story of an obscure kernel bug, how we missed it, and how we’re still recovering from the effect

I should preface this by saying that, generally, I like virtual machines.
I have 5 actual servers doing actual things- everything else is a VM in a racked bunch of servers hosted at Telecity in east London.

Generally, these servers are catered for with two uncontested fibre-to-the-rack lines which are layer-4 DDoS scrubbed and redundant power from two seperate generators and dirty feeds. – believe me when I say, no expense is spared on that rack, it’s where 90% of my budget goes and as well it should, given it’s the core business of the company.
I should also preface this by saying ubuntu has held hatred of mine for some time- given we had a development server here in the office and it failed due to a name change of lvm2 to lvm in initramfs causing our...

Continue reading →