#InfoSec - A year memoir of Fukushima 3-11 | A geek's diaster diary

It is almost a year from 3-11 Quake and Tsunami disaster in Fukushima. I was there, I saw and remembered it well. Yesterday I was attending Defcon and discussing this with the guy sat next to me. He urged me to write it, and this is it, I dedicate this writing to all 3-11 quake victim and to fellow IT/security engineers everywhere to let you know the actual feeling of what happened on-the-site.

I was working in the Fukushima educational center for trouble shooting some network security filtration scheme of our solution when earthquake started on March 11th last year. When it was started 6(six) of us was inside of the secure data center. I was analyzing the SYN packet sent and received between proxy layer and core switch within so I was standing in front of one rack. The server's racks itself contains of 3 blocks of 20U racks of 10feet long. Shortly, it happened so fast yet every glimpse that I saw was recorded & formatted in my brain in a sequence of slow motion so well so I can tell you the exact detail of it. Those were nights I woke up with the nightmare of these visions and this is the first time ever I finally dare to write it.

When it was started, it was just like the previous smaller quakes, the horizontal movement began slowly followed by the strange deep voice of "Boom, boom, boom..!" coming from the center of the earth, which went faster, then the vertical rocks starting to merge with it. We were stunned for at least at the first 10secs, afterward one of us said "This one is getting harder isn't it?", and that was the word that triggering our first action. About 20Us size server's rack (around 2meter height) was rocked so hard, and every block of rack bumped to the space limit safety bare-bone (actually it was made with slack of 1 meter size) with the loud voice, the balls inside of the rack's conjunctions was popped out like a bullet, and some server's, routers, switch's part was starting to fall (or FLY will be more like it..). Still, at that time we aimed to save the service first, four of the engineers run and hold the rack by their both hands. the rest of us starting the shutdown procedure.

At that time the quake started to rock us with a hoopla-like movement, I made decision, to yell out loud to others to leave/forget the racks & to just run out of the data center room and out of the building a.s.a.p; can't remember much; but I was saying those with grabbing 2 engineers and pull them all to the exit door.

At that time the call coming from Tokyo and was my wife reporting the same condition (3seconds established call), well, she heard my yell to other fellow engineer in the data center and made her running out of the house too, good for her (LOL). Meanwhile inside the IDC the electricity starting to black out. This causing us cannot get out of the room for the authentication card cannot be read due to power off (surprisingly the backup generator NOT running). Jammed at the door, I took a glimpse to the servers and made sure the UPS running and holds the services for the shutdown procedure of every clustered servers. At that time the mobile service of the earthquake alert sounds started to buzzed (Those alert were a way too slow, means nothing to us at that moment...), starting to wonder what would happen next if we don't get out that room..

Finally using the manual step the door finally opened up and we started to just run for the building door, on the way there through the exit door, at that time the building was rocked so hard so we run and managed not to fall at the same time. I saw 5of us went out to the exit door, yet one person went back to the IDC to get the battery so I chased him back there.

We managed to fly off the exit door and at the time the concrete block of the building was starting to fall to the parking lot and exit door's nearby.. We were lucky I guess. The first sight I saw after stepping out of the building was the concrete block falls all over, some car wrecked by those in the parking area and right in front of us there was a slim 5th floor building was rocked side-to-side until 25degree each.. which throwing every antennas parts like rockets in every swing. That time I realized that I carried my bag, but I didn't even know when was the time I grabbed it. We were gathered there until 15:40 at that time, before I decided to go to Fukushima station, to find a way home. The worst part of it is the news, the tsunami news can be viewed by the TV inside of the cars.., we were thinking of the end of the world that time.

One of the members was driving me half way to the station, which was a hard stuff since no traffic light works and many panic drivers thinking only their own priority, I continued to go to biggest train station by foot and reach there to find about almost 200peoples standing in the front of the station building which cannot be entered since the quakes still came like waves with the quite big scale (within 4 to 5 each).

To make our disaster perfect, the snow started to fall. I saw some office workers without the coat out there too. Can't use any mobile devices, no phone calls. I checked every internet connection I have, after some rapid dial here and there found that eMobile antenna connection can be used, yet there was a problem in resolving DNS, hurried putting OpenDNS & Google's DNS IP in every resolve.conf of my devices, afterward I tried to check my house IP but got timed-out. So does the office network and tokyo IDC. At that time Skype call came into my mind. After the 10th effort connecting to Skype network I finally got it. I was thinking to make connection alive as long as possible. I tried to make phone call, avoiding mobile ones, and successfully made some. Every people in there seeing me amazed, so I was explaining use the Wifi and Skype and you might made a call, and they started to tried it.

Meanwhile, my PC was connecting to eMobile and I can get the radio internet tune, people started to gather. At that time the first nuclear plant explosion news came into our ears I guess. I was a kind of news center, and keep on doing that until my PC went out of battery. Afterward, police station nearby managed to get taxi drivers to pick them as passengers (5person in each taxi) and evacuate them from station to the nearest residence area (usually there were plenty of taxicabs in front of station but the quake made them runaway), while I was staying there & start arranging my way home, which will be another long story that took me 3days to get back home.

The moral of this true story is; Number One, in disaster like this: human is first & machine is next! As an one of UNIXmen we are pledged to make a service lasts as long as possible, maybe longer than human's lifetime, that was why in BSD we said Daemon to services. BUT, in a practical way, no human means no service. We should say over and over that saving a service is important yet is up the some certain condition of a disaster, when the condition goes near to live or death, just drop everything and run off. Damage will occur and so be it, because without your life the damage cannot be fixed. P.S.: If the service is involving lifeline level of a lot of people, maybe it will worth to die for, and that's is a different one. This priority will be hardly remembered during the disaster, so next tip is important..

Number two is, to train yourself to handle this kind of situation! We should pay more attention to the disaster simulation handling. No person who born with the gift for handling these pressure, to train yourself to act automatically in the disaster is a must! Which is not just a merely training but think & imagine it as real as possible. I am telling you, no brain can overcame fears caused by these panic & stress, and the cure of it is your rehearsal effort.

Number three, be resourceful in media connectivity, as much as possible in your daily life (internet, radio, TV, anything!). As engineer you SHOULD at least know that you can be connected to network as many way as possible. If you have a hand-phone make sure it has FM/Radio on it, if you have smart-phone be sure to have WiFi function on it, If you use Wifi, make sure you have some services to connect to & do not depend into one connectivity. If you have latest smart phone, make sure it can do tether on it. This diversification of connectivity know-how will be your weapon to survive in disaster. And don't forget your bag. You have many surviving tools in it, more than you know it, you might need it later.

---
Based on personal experience,
Twitter: @unixfreaxjp
*) all of the picture attached is to be viewed publicly in some media & all rights are copyrighted & reserved.