Jan Harasym

Designing highly scalable/resilient infrastructure by day; running hacker communities by night.

Read this first

FreeBSD; I guess we weren’t destined to be.

I always write my blog articles, especially my opinion pieces as if, you, the reader, do not know me at all. Unfortunately- today I must break that tradition.

You may remember I wrote a post a long time ago detailing why darkscience no longer operates with github - You may remember that due to the hamfisted throat-ramming of systemd; darkscience transitioned to using FreeBSD for most of its infrastructure needs.

Well, now, these two things are, apparently, linked.

For those that don’t know the FreeBSD community has recently come out with a controversial Code-of-Conduct. I don’t wish to go into the details too much, other than that it was taken from a political source and, among other things, puts bans on virtual hugs and anything that

“reinforce[s] systemic oppression [..]”


“Unwelcome comments regarding a person’s lifestyle choices and practices, including those related to fo...

Continue reading →

RacAdmin Quick and dirty cheatsheet

iDRAC racadm quick and dirty cheatsheet. racadm command can be issues via iDRAC/CMC/OS if svradmin-racadm is installed. Also you can specify -h option to access remote servers RAC as long as you have network access. Also if you are having problems with racadm “Failed to initialize transport” install openssl-devel. full documentation for iDRAC7 can be found here.

% Get all iDRAC settings in a file

racadm get -f config.txt

If you like you can change the contents of config.txt and apply it back to iDRAC

racadm set -f config.txt

% Set password for root user

racadm set iDRAC.Users.2.Password PASSWORD"

% List all ssh keys for root user

racadm sshpkauth -i 2 -v -k all

% Add ssh key to root user

racadm sshpkauth -i 2 -k 1 "CONTENTS OF PUBLIC KEY"

% Delete ssh key for root user

racadm sshpkauth -i 2 -d -k 1

% Get iDRAC IP config

racadm getniccfg
racadm get iDRAC.NIC

% set iDRAC IP


Continue reading →

Follow Up: Wildcard TLS Certificates

Definition of WildCard

I wrote an article some time ago in a fit of anger about people continually bashing LetsEncrypt for not supporting wildcard certificates.

Why was I angry? Well my original post is here and it’s about as ranty as
you would expect from me. In it, I call people lazy and falsely attribute the
fact that SSL Certificate Authorities will not insure their wildcard certificates as a reason to avoid them. (I implied customer insurance would inform business decision making).

I figured since LetsEncrypt have caved and started supporting wildcard certs I should follow up and touch base more objectively with the reasons I feel it’s a poor practice for your users. There are reasons to use wildcard SSL certs and I’ll touch on those too.

 Revocation Issue

Revocation is, unfortunately, in 2017, not a solved problem.

OCSP is still susceptible to this attack since 2009.

Revocation is amplified on...

Continue reading →

Trusting the user; they know what language they speak.

As a digital nomad one of the most difficult things to overcome is language barriers, most people default to English as a lingua franca but computers can’t be so easily coerced if there is a “smart” website which geo-locates your IP and serves you a localised webpage and no option to disable it.

Surely there is an easier way to ascertain browser localisation.

Maybe something in the http headers that browsers always send..

Maybe it could be called something like Accept-Language as in “this browser accepts these languages”…

oh, it exists- what luck!

Screenshot from 2017-05-26 10-10-05.png

Maybe we can use this instead?

View →

[OP] The decay of reliable infrastructure.

I started writing this six months ago as a terrible opinion piece surrounding cloud computing in general. In it, I shredded many cloud conceptions regarding scalability/cost and highlighted the fact that not all needs fit in cloud sized containers. (I’m sure there is a docker joke in there somewhere).

Alt text?

However, my problems ultimately boiled down into two categories;

A) The people who force the statement: “[we] need to cloud!”. (usually directors being courted by the amazon sales team)


B) Cloud providers reliability. (or, their lack of it, and pushing of issues up the stack)

The former is a company problem, and isn’t an issue with the concept of the cloud at all, it’s more of a nail->hammer issue. However! The latter is certainly where most of my gripes lie.

I was #triggered when I read an internal memo from our Director of Infrastructure at Ubisoft- and while his post and name...

Continue reading →

On the importance of self-hosted backups.

A long time ago I built a pretty big storage computer (16TB) which I built because SSDs at the time were pretty small and most laptops came with only a single possible SATA drive bay for storage.

I’m also quite a large proponent of self-hosting/federation and things of this nature as evidenced by our gitlab, mail service and IRC. However, most people assume this huge storage unit is for piracy. In fact Sweden has a tax on storage devices because they may be used to store pirated media, that is also true in other EU countries such as the Netherlands [PDF]. But my unit is not for piracy- despite there being digital creative works on there I prefer to buy all my music through iTunes.

iTunes on iOS

Why? Because then it’s synced to all of my devices, I don’t worry about ownership or getting it synced, or worrying about the bitrate/collecting albumart/removing weird watermarks people add… it’s simple...

Continue reading →

Reminder: Keep your IRC Email up to date

If you’ve registered with NickServ on darkscience within the last few years then you’ll have used an email address and we’ll have sent you a mail to verify it. That will probably be the last time you heard from us…

…until you forget your password and find yourself unable to identify to your account. When that happens we can send an email (only to that same address) to verify your identify and reset your password.

You aren’t stuck with the email you originally used though! I’d very strongly recommend you take 5 minutes to double check the set email address is current, especially in light of recent service closures. You don’t need access to your old inbox to change your registered email, just your NickServ password.

To view the current state of your account, while identified type:

/msg nickserv info

If you’d like to then change the registered email address, first…

/msg nickserv set...

Continue reading →

Elasticsearch Notes

Elasticsearch is 2 components.

  • Elasticsearch: clustering engine and REST API
  • Lucene: Search backend. (indexes are always raw lucene)

You need to understand how both work;


 Index Merges

This video displays how index merges occur:
Indexing Mediawiki

Basically when you have enough segments that can be grouped they will be vacuumed and merged.


 Memory Pressure/Heap:

If you monitor the total memory used on the JVM you will typically see a sawtooth pattern where the memory usage steadily increases and then drops suddenly.


The reason for this sawtooth pattern is that the JVM continously needs to allocate memory on the heap as new objects are created as a part of the normal program execution. Most of these objects are however short lived and quickly become available for collection by the garbage collector. When the garbage collector finishes you’ll see a drop on the memory...

Continue reading →

don’t pipe curl to bash

Unless you haven’t been installing developer focused 3rd party software recently, you will probably have seen the following command line used as a suggested way of installing a particular software package direct from the web:

curl -s http://example.com/install.sh | sh

This post is not here to debate whether or not this is a good idea but rather to make those that use this pattern aware of a non-obvious flaw, aside from all the obvious issues with piping 3rd party data directly into your shell. There have been countless discussions on this method and one argument for it has always been transparency - as in, you can simply check the script by opening it in your browser before piping it to bash via curl.

This post is here to a) show that this level of trust can be hijacked and b) to provide an easy way of protecting yourself when you wish to install via curl.
Proof of concept...

Continue reading →

Tor @ DarkScience

For completeness I’m going to write this post as if you know nothing about me or Tor. If you know about DarkScience and Tor you can safely skip to here.


Tor, or more formerly “The Onion Router” is a method of decentralised VPN, it’s more commonly referred to as a “Anonymity network” and that would be very apt. The design of Tor is an encrypted mesh network operating over the top of the regular internet, there are intermediaries which have no concept of the data, where it came from or where it’s headed- only it’s next hop in the chain. There are entry points who only know where the data came from but not where it’s going or what it is, and more famously there are exits which take traffic originating from somewhere in the tor network and allow it to enter the public internet.

Tor, typically is rather slow, and it’s possible to address servers and services without ever leaving tor...

Continue reading →