Jan Harasym

Unix Systems Admin by day, netadmin and Systems Architect by night.

Read this first

Trusting the user; they know what language they speak.

As a digital nomad one of the most difficult things to overcome is language barriers, most people default to English as a lingua franca but computers can’t be so easily coerced if there is a “smart” website which geo-locates your IP and serves you a localised webpage and no option to disable it.

Surely there is an easier way to ascertain browser localisation.

Maybe something in the http headers that browsers always send..

Maybe it could be called something like Accept-Language as in “this browser accepts these languages”…

oh, it exists- what luck!

Screenshot from 2017-05-26 10-10-05.png

Maybe we can use this instead?

Continue reading →

[OP] The decay of reliable infrastructure.

I started writing this six months ago as a terrible opinion piece surrounding cloud computing in general. In it, I shredded many cloud conceptions regarding scalability/cost and highlighted the fact that not all needs fit in cloud sized containers. (I’m sure there is a docker joke in there somewhere).

Alt text?

However, my problems ultimately boiled down into two categories;

A) The people who force the statement: “[we] need to cloud!”. (usually directors being courted by the amazon sales team)


B) Cloud providers reliability. (or, their lack of it, and pushing of issues up the stack)

The former is a company problem, and isn’t an issue with the concept of the cloud at all, it’s more of a nail->hammer issue. However! The latter is certainly where most of my gripes lie.

I was #triggered when I read an internal memo from our Director of Infrastructure at Ubisoft- and while his post and name

Continue reading →

On the importance of self-hosted backups.

A long time ago I built a pretty big storage computer (16TB) which I built because SSDs at the time were pretty small and most laptops came with only a single possible SATA drive bay for storage.

I’m also quite a large proponent of self-hosting/federation and things of this nature as evidenced by our gitlab, mail service and IRC. However, most people assume this huge storage unit is for piracy. In fact Sweden has a tax on storage devices because they may be used to store pirated media, that is also true in other EU countries such as the Netherlands [PDF]. But my unit is not for piracy- despite there being digital creative works on there I prefer to buy all my music through iTunes.

iTunes on iOS

Why? Because then it’s synced to all of my devices, I don’t worry about ownership or getting it synced, or worrying about the bitrate/collecting albumart/removing weird watermarks people add… it’s simple

Continue reading →

Reminder: Keep your IRC Email up to date

If you’ve registered with NickServ on darkscience within the last few years then you’ll have used an email address and we’ll have sent you a mail to verify it. That will probably be the last time you heard from us…

…until you forget your password and find yourself unable to identify to your account. When that happens we can send an email (only to that same address) to verify your identify and reset your password.

You aren’t stuck with the email you originally used though! I’d very strongly recommend you take 5 minutes to double check the set email address is current, especially in light of recent service closures. You don’t need access to your old inbox to change your registered email, just your NickServ password.

To view the current state of your account, while identified type:

/msg nickserv info

If you’d like to then change the registered email address, first…

/msg nickserv set

Continue reading →

Elasticsearch Notes

Elasticsearch is 2 components.

  • Elasticsearch: clustering engine and REST API
  • Lucene: Search backend. (indexes are always raw lucene)

You need to understand how both work;


 Index Merges

This video displays how index merges occur:
Indexing Mediawiki

Basically when you have enough segments that can be grouped they will be vacuumed and merged.


 Memory Pressure/Heap:

If you monitor the total memory used on the JVM you will typically see a sawtooth pattern where the memory usage steadily increases and then drops suddenly.


The reason for this sawtooth pattern is that the JVM continously needs to allocate memory on the heap as new objects are created as a part of the normal program execution. Most of these objects are however short lived and quickly become available for collection by the garbage collector. When the garbage collector finishes you’ll see a drop on the memory

Continue reading →

don’t pipe curl to bash

Unless you haven’t been installing developer focused 3rd party software recently, you will probably have seen the following command line used as a suggested way of installing a particular software package direct from the web:

curl -s http://example.com/install.sh | sh

This post is not here to debate whether or not this is a good idea but rather to make those that use this pattern aware of a non-obvious flaw, aside from all the obvious issues with piping 3rd party data directly into your shell. There have been countless discussions on this method and one argument for it has always been transparency - as in, you can simply check the script by opening it in your browser before piping it to bash via curl.

This post is here to a) show that this level of trust can be hijacked and b) to provide an easy way of protecting yourself when you wish to install via curl.
Proof of concept

Continue reading →

Tor @ DarkScience

For completeness I’m going to write this post as if you know nothing about me or Tor. If you know about DarkScience and Tor you can safely skip to here.


Tor, or more formerly “The Onion Router” is a method of decentralised VPN, it’s more commonly referred to as a “Anonymity network” and that would be very apt. The design of Tor is an encrypted mesh network operating over the top of the regular internet, there are intermediaries which have no concept of the data, where it came from or where it’s headed- only it’s next hop in the chain. There are entry points who only know where the data came from but not where it’s going or what it is, and more famously there are exits which take traffic originating from somewhere in the tor network and allow it to enter the public internet.

Tor, typically is rather slow, and it’s possible to address servers and services without ever leaving tor

Continue reading →

Please stop advocating wildcard certificates.

Ever since I started “doing the computer stuff” I’ve been a bit wary of SSL/TLS.
It’s very easy to get wrong and revocation is not a solved problem no matter which security vendor is trying to push for sales.

There is, however, a strong urge to do things as easily as possible. For most people using SSL/TLS this becomes:

  • Use openssl commands from a 5 year old blog to generate a CSR.
  • Paste your CSR onto some web form.
  • Copy/Paste ciphers/config from some other website.
  • Ensure usage of wildcard certs. so you never have to do this pain again.

Some of these steps are worse than others, there’s almost no risk of using old openssl commands (except smaller keysize), but if you have an old list of cipher suites you’ll probably be using a deprecated cipher- and your clients data may be vulnerable in transit.

The worst one by far is the wildcard certificate; and this is for mostly obvious

Continue reading →

Dropping filesystem caches.

By writing to /proc/sys/vm/drop_caches the linux kernel will drop clean caches, dentries and inodes from memory, causing that memory to become free. (yay!)

To free pagecache:

echo 1 > /proc/sys/vm/drop_caches

To free dentries and inodes:

echo 2 > /proc/sys/vm/drop_caches

To free pagecache, dentries and inodes:

echo 3 > /proc/sys/vm/drop_caches

As this is a non-destructive operation, and dirty objects are not freeable, the user should run “sync” first in order to make sure all cached objects are freed.

This tunable was added in 2.6.16.

It’s pretty handy when trying to benchmark disks.

Sidenote: on FreeBSD this is impossible, you have to unmount the disk and remount it.

Continue reading →

Moving away from GitHub.

 The tale of centralised source code management.


For a long time I’ve been a user of github, for personal public projects it has served me well enough and “github is the home for everything modern” has been sort of ingrained into my head for some time.

For private things we at darkscience and at libsecure had used Atlassian bitbucket, since they have a free tier that allows only private repos, and github has the same thing but about public repos, they complimented each other and our objectives perfectly. There was a minor hassle of course, but generally for a frugal group of people such as us, it was perfect.

 And then, the shitstorm;

ok, not a real shitstorm, more like “increasing amounts of discomfort”, github were beginning to censor and take down random repositories.

It’s not that we do anything particularly bad at darkscience but we generally err on the side of

Continue reading →