An interesting day in #Barcelona it was...

I expected it to be one as I was scheduled to do a presentation with the "Autoritat Catalana de Protecció de Dades" on the risk of S3 buckets and other data leaks. For me it was a special presentation because the first time in my life I would do a presentation in Spanish. Put a lot of preparation into it but was nervous anyway. But it went fine and we wrapped it up at about 12:00 local time.

As we (Directora Meritxell Borràs i Solé and me) were debriefing, suddenly the lights went out. I expected a small localized hiccup, so we didn't think much about it and started our journey to the scheduled lunch. A taxi was waiting for us.

1/7

On the way to the taxi we got the news, the outage was city wide. That was giving me bit of a worry for our lunch as I don't expect restaurants to keep working without electricity. This turned out to be right and wrong at the same time.

Traffic in Barcelona is a mess at best times and these weren't. But a total collapse was avoided as most traffic lights kept working, a thing a reliability enthusiast like me noted positively. But I also saw a fraying of the mobile network that frankly shocked me. I would have expected problems, but not that soon. During our 15min drive to the restaurant I had everything from excellent connectivity to no reception at all.

By the time we arrived at the restaurant, it was clear that something big was happening. We had reports of outages from Portugal, all over Spain and from Southern France at well.

2/7

As it turned out (after four floors upwards on ramps), the restaurant was completely out of commission. While they still had gass to cook, no light made the kitchen inoperable and the beverage dispenser were out of commission as well. So we went four floors down again.

At that point I presented my opinion that we would be dealing with hours of outage. Small outages can usually be fixed quickly, but nation wide is a totally different game. I said that I would be very happy, if we had electricity before sundown. I recommended to my hosts to make their way home as traffic would not get better and that I needed not to be taken care of. My hotel was about a mile away from the restaurant so we split up. This was about 13:30.

At that point I started some actions of my own:

  • Phone was set to power saving mode and brightness reduced to "barely legible"
  • At the next open store I bought non-alcoholic drinks and food to get me through two days
  • I asked my colleagues in the SOC to go to an higher alert level

I am kind of a pessimist in such things...

3/7

At that point I had only terrible mobile phone coverage. My evaluation showed:

  • I could do outgoing calls in terrible quality, but not receive incoming calls
  • Text messages could neither be sent nor received
  • Internet was completely down

I would love to read the post mortem from the mobile phone companies. That should not have happened within the first hour of the outage and is a big "no no" from my PoV.

As I was already three weeks in Barcelona, I noticed an unusual pedestrian traffic pattern. There were a lot more people on the streets than there were usually at this time of day. All the large stores and restaurants had closed and the patrons were pushed to the streets. People were no longer using their phones in the usual way. Usage in unsual ways was mostly shouting or staring frustrated at it.

4/7

@masek Also in germany Internet is gone really really quickly if the power went down. Especially for discount carrieres like O2. We have some regular small power outages here in the village (most 30m-2h), as the local network is strained by Tiefbau (Deutsche Glasfaser) and quick adoption of PV.

The thing going away the first is the Internet for low cost carriers. For all others, the bandwidth will go down to pre-LTE levels.

I suspect this is some optimization for the batteries.

@hikhvar At home, I have three carriers (Fiber, DSL, Mobile) with an automatic fail-over. I hope at least one will work.

With the PV, I can use my battery as big backup power battery.

@masek From my experience, non of them work if there is a complete power outtage in the village. Both Fiber and vDSL are immediatly dead. Mobile becomes shitty very soon.

I have a PV battery with backup capabilities. Often I use that to get me some coffee and enjoy the time for some local-non-internet programming work. Everything else is impossible.

@hikhvar @masek what is PV?

@heleenkuiper

German abbreviation for photovoltaics.

@masek

@hikhvar @masek and that is a solar panel?

@heleenkuiper
Yes, I have a home solar panel installation and a battery which is managed by my solar inverter. The inverter can draw power from the battery to feed my house, if there is no sun. What is more, this even works, if the grid is offline. Many older and cheap inverter/battery combinations will not generate any power if the grid is offline as it was in spain yesterday.

@masek

@hikhvar @heleenkuiper @masek first time hearing about regular blackouts in Germany. Why is it happening?

@emaksovalec

This is very locally to our village and most outtages are caused by construction work for the new fiber to the home network. They have often hit some cables. On time this took out 2 transformers, resulting in up to 3h ouf power outtage. Also there are often (2-3 times a year) outtages of 5-10 minutes. I suspect they have to adopt the grid to PV. We got ALOT of new PV in the village in the last 2 years. And we have a lot of sun here.

@heleenkuiper @masek