Administrivia

Users of blakjak.net may have noticed the following:

1) A couple of brief network outages over the last few days. These are tied to our colocation provider and their upstreams, and are being investigated.

2) Mail delivery delay overnight last night. This is due to a quirk of the mail server and some maintenance performed last night. No mail will have been lost - all mail accepted - but delivery to local mailboxes (and relays for those using forwarding) would've been held up until shortly before 9am. Sorry for any inconvenience.

Cheers

... crashed shortly before 8pm tonight and service restored around midnight.
This time at least it was a genuine server crash :-|

In other news, however, the replacement hardware is currently under test and is looking good. David and I will be working to get it sorted in the next couple of weeks and installed in the weeks following that.

Stay tuned, will notify people affected with regards to any (scheduled!) outages required.

I know, it's a broken record...

A network issue caused an outage this afternoon for a few hours.
Our provider helped us sort it out and there should be no long lasting issues.
Sorry for the inconvenience.

On the evening of Saturday 29 December your humble admin was doing some admin work in preparation for some upcoming Domain Name Changes when he noted intrusion attempts coming in near-real-time from a foreign host attempting to break one of his daemons (services).

He took action - a little too much.
Thanks to some not-so-well-thought-out software syntax, maverick.blakjak.net went offline (for TCP purposes) and remained offline until our beloved Colo Provider Helix returned from holiday to reboot the server. ;-) (How's that for timing? Pick the one interval where there's noone handy to fix it...)

Total Down time was around 5 days and 18 hours. Fortunately, having come in under 7 days (barely) there should be no loss of mail (messages would've queued and will have been delivered over the course of this afternoon in the usual routine of retrys per RFC) and outside of the fact that things were a bit broken in the web department, services are back to how they were.

(If you do spot any weirdness, those hosted here should know how to get hold of me. Else hit the contact form above.)

What do we learn from this experience? When screwing with your firewalls, think real hard about the command you're entering before you submit it!!

The only thing to be grateful for was that due to the timing (holidays), little business was disrupted.
Unfortunately the timing also contributed to the delay in restoring service (which would otherwise have only been a few hours lost.) Lesson learnt.

This experience also has me working harder on the sequel to maverick; Stay tuned. (yeah, i keep saying that... but having broken all records for service interruption, I'm very interested making sure an event like this doesn't happen again.... !)

Sites using a MySQL Backend on BlakJak.net experienced an outage this evening, between approx 1750 and 2025hrs. I'm still investigating the why but services appear to be restored for the moment. Sorry for the inconvenience.

Apologies for any inconvenience caused by a loss of services from blakjak.net between ~0100 and ~1200hrs today, and thanks to Helix for restoring services for us as quickly as they did once notified.

Any remaining problems please let me know...

Just a quick note, its observed that blakjak.net is unroutable for some networks in Wellington at present, so just to indicate that we're aware of it and have advised our hosting provider and the relevant upstreams...

Sorry for any inconvenience - but this one is out of our hands, we're waiting on advice from Helix...

Cheers
BJ

Yeah. Things are looking a bit funny right now.
Whilst doing a routine upgrade of Drupal Core I managed to break my main operational theme.

So things look a bit funky til I fix it. Sorry for anyone who finds it a pain...

(expect that'll be mainly me...)

Our apologies for any loss of service experienced between approx 0100hrs and around lunchtime today. Our server crashed, for reasons unknown at this stage (actually, we have a lead on it, but it appears to be largely random...)

The server was rebooted and services appear to be working OK, with no apparent data loss.

Stay tuned for changes to the BlakJak.net environment...

Over the last couple of weeks it has been noted twice - once by myself and once by 'someone else' (i'm sure the person who told me wasn't the person affected!) - that rbl.blakjak.net has been catching a couple of errant netblocks as being 'China/Korea' blocks.

Those who use mail services hosted through me will be aware I have been using a web resource to compile a list of IP networks in use in China and Korea, and actively block their SMTP traffic - to reduce spam.

Anycase investigation shows that a subnet in use by InspireNet has been incorrectly identified as being Chinese. So some users of Inspire would've had trouble mailing accounts hosted here.

In light of this I have had that component of the RBL disabled; blocks I have manually added are retained in the system but the auto SinoKorea block is no longer in effect. I've whitelisted the range concerned - so I could turn it back on - but i'm also enquiring as to the accuracy of the source info.

Greylisting should continue to make a dent (The SinoKorea block pre-dates Greylisting, meaning potentially, it could now be redundant), however, would be interested in hearing from any people whos mail hosted @ BlakJak.net shows any changes to spam behavior / volume as a result of the actions taken today. Feedback will help us determine how best to run the mail systems going forward.

Cheers
BJ.

Syndicate content