It Came from the Long Beach Comic & Horror Con 2013

My lack of a plan for this year's con made the show floor less satisfying, but the panels were great: Marv Wolfman, Scott Lobdell, Mark Waid, Young Justice.

https://journal.kvibber.com/2013/11/lbchc-2013/

Oddly Specific Advertising

A Casper billboard on PCH advertises the 'perfect mattress for furries.' I assume it's part of a series. Either that or they're into really niche marketing.

https://journal.kvibber.com/2015/11/niche-marketing/

Silly facts about me, and maybe you want to answer these questions too? Let me know if you do!

#blogging #creativeblogger #blog #wordpress #stillgoingstrong #nablopomo

https://www.ihanna.nu/blog/2026/03/get-to-know-ihanna/

Get to know iHanna in 15 quick questions | iHannas Blog

I have gotten so many new subscribers to the blog (hiii there!) that I thought I'd post a silly little introduction post today. A get to know me, iHanna, in a few random questions about this and that. Not in depth or anything, just a bit of fun because I don't do things like this

iHannas Blog | creativity in everyday life

The 360 Selfie

https://shkspr.mobi/blog/2019/11/the-360-selfie/

I'll admit it - I never really got the selfie. I was trapped in the mistake that they were a mere narcissistic folly. I couldn't have been more wrong. The more I travel the world, the more I see a change in people's behaviour around monuments. It used to be people jostling to get the one perfect shot of a thing. Even if they got the perfect shot, they never looked at the result. Why?

People want to take a photo like this:

But it often ends up coming out like this:

That's not intended to be snarky. Most of us don't have the time, skill, or equipment to take stunning photographs. But that's OK! The best photographs of mere objects have already been taken. The photo that no-one else has ever taken is of you standing there!

That's what I love about selfies! A photo of a monument is just that. It shows you were there, once, kinda. But a selfie proves that you were there! I think there's something delightful about saying "This is me! Here I am!"

So, here are a couple of 360 selfies from our recent trips. They're not the greatest photos in the world - they're not static shots of a thing - but they are of us and the memories we made.

Rome

Hong Kong

Australia

#360 #LG360 #meta #NaBloPoMo #photography #photos
The 360 Selfie

I'll admit it - I never really got the selfie. I was trapped in the mistake that they were a mere narcissistic folly. I couldn't have been more wrong. The more I travel the world, the more I see a change in people's behaviour around monuments. It used to be people jostling to get the one perfect shot of a thing. Even if they got the perfect shot, they never looked at the result. Why? People…

Terence Eden’s Blog

New blog post: “Third Complete November”.

https://blog.kizu.dev/third-complete-november/

For the third time, third year in a row, I managed to write a post for every day of November.

I was tired at the end, and postponed writing many posts until the end of the day, but it felt easier than the last two previous years.

#NaBloPoMo #IndieWeb

Third Complete November

For the third time, third year in a row, I managed to write a post for every day of November. I was tired at the end, and postponed writing many posts until the end of the day, but it felt easier than the last two previous years.

HOWTO: Preserving BarCamps

https://shkspr.mobi/blog/2010/11/howto-preserving-barcamps/

Oh! Woe is me! I can't make BarCampLondon 8. This will be the first BarCamp I've not been able to get to in ages.Following hashtags is fun, if a little disjointed. What I really need is some way I could attend virtually.....

BarCamp London

@barcamplondon

Replying to @edent@edent if you provide the cameras, the 20 person film / editing crew along with some budget sure ;) Its probably a bit much sorry! ❤️ 0💬 0♻️ 023:00 - Wed 10 November 2010

Does It Need To Be So Hard?

We don't need expensive crews. We have each other! The very idea of BarCamps is that they should be self-organised. So, if you're going to BCL8 this year, here's what I would like you to do...

  • Got a smartphone or a flip camcorder? Video the sessions you're in.
  • Got a less-than-smart-phone or a dictaphone? Record the audio of the sessions you run or participate in.
  • Got neither? Offer to hold someone else's tech while they present.

Before long, every single session will have coverage of some kind and us poor sods who can't make it will still be able to vicariously share in the pleasure of London's greatest Unconference.

Video

There are loads of places to stick video up on the web. But most services have time and size restrictions or want you to pay to store your video. YouTube is limited to 15 minutes (no good for 30 minute BarCamp sessions) and Vimeo is limited to 500MB per week (no good if you've filmed in HD).

Here are two free services which I think will suit the BarCamp Crowd.

Archive.org

Archive.org is

building a digital library of Internet sites and other cultural artifacts in digital form. Like a paper library, we provide free access to researchers, historians, scholars, and the general public.

I believe this to be the best place to store the record of our culture.Here's a video of me running a session at BarCamp Brighton. The video is hosted by Archive.org and can be downloaded in several formats.

Videos can be any length and any format. They'll be transcoded to MP4 and Ogg Video. Best of all, video can be uploaded via FTP - which makes queuing large uploads very easy. You can, of course, upload via your browser.

One point to note, videos have to be CreativeCommons or Public Domain.

There are already around 40 BarCamp videos on Archive.org.

Qik

If you have the bandwidth, Qik is the ideal place to stream and store video. I've found the application can be a little hit-and-miss - especially if WiFi coverage gets congested.Here's my presentation from BarCamp Transparency.

(Qik videos autoplay - click on the screenshot to see the video)

Streaming Services

There are loads of sites which will let you stream video from your laptop's webcam. Please - whatever service you use - make sure that it is also recording the stream. It greatly saddens me that the presentations from awesome events like CharityHackDay are lost forever. They were streamed and then promptly vanished.

Audio

Video can be overkill. It takes a lot of effort to upload and just as much to download. It's often simpler just to record audio. Even the most basic phones tend to have a "voice record" function. Once you've got the audio, there are a number of places you can put it.

AudioBoo

Everyone loves AudioBoo. It's a great way of saving audio - although it's currently only available on iPhone. If you don't have an iPhone, you can upload any sound file you like to their website and they'll convert and host it.

Here is a recording of my session at BarCamp Brighton.

🔊 Location Based QR Codes - Introducing http://xmts.mobi/
🎤 edent

💾 Download this audio file.

I decided to separate the main body of the talk from the discussion which followed afterwards.

🔊 Discussion: Introducing http://xmts.mobi/
🎤 edent

💾 Download this audio file.

Just personal preference.

Archive.org

There's a whole bunch of BarCamp audio already up on Archive.org.

Audacity

If you want to edit your audio, I highly recommend Audacity. It's free, open source, and works on Linux, Mac and Windows.

Collating

Ok, so every session is now recorded and stored somewhere on the Internet. How do we go about organising this mass of video and audio?

Let's look at how some other conferences have done it.

OpenTech

One of the great things about OpenTech was the effort put in by the organisers to ensure every talk was preserved on wax cylinders mp3.The session page is really well laid out and has links to audio and, in some cases, slides.

But what if you can't be bothered or don't have the time to manage a web page?

Lanyrd

Currently the darling of the conference scene, Lanyrd easily allows you to add "coverage" to sessions. You can attach notes, handouts, audio, video, slides, photos or links.

Privacy

Terence Eden is on Mastodon

@edent

Replying to @barcamplondon@barcamplondon @cbetta @tommorris doesn't need anything expensive. Just one person in each session recording on their phone / flip. ❤️ 1💬 0♻️ 023:25 - Wed 10 November 2010

Richard Johnson

@chichard41

Replying to @edent@edent I recorded some videos of #bcl7 but they're massive to upload, some on my vimeo. Or some are "what's said in barcamp, stays in..." ❤️ 0💬 0♻️ 023:49 - Wed 10 November 2010
  • Some people won't want their sessions recorded. Please ask presenters to check that they're happy to be immortalised.
  • Everyone hates the sound of their own voice - do let people in the audience know if you're going to be recording.
  • When in doubt, edit it out.

A Plea

I really want to attend BarCamp London. And BarCamp Manchester. And all the other BarCamps which I can't get to.

Please - if you can - record your session and upload it somewhere. Even a poorly shot, noisy, unedited video is better than none.

This is our digital heritage. We should be proud of what we create and ensure that it is available for posterity.

#audio #barcamp #digital #nablopomo #preservation #video

HOWTO: Preserving BarCamps

Oh! Woe is me! I can't make BarCampLondon 8. This will be the first BarCamp I've not been able to get to in ages. Following hashtags is fun, if a little disjointed. What I really need is some way I could attend virtually..... .social-embed {all: unset;display: block;}.social-embed * {all: unset;display: revert;}.social-embed::after {all: unset;}.social-embed::before {all:…

Terence Eden’s Blog

Death Etiquette For The Social Media Age

https://shkspr.mobi/blog/2016/11/death-etiquette-for-the-social-media-age/

My friend died last month. When I was a lad, it seemed that I was always going to funerals for some ancient relative. Everyone looks old when you're a kid. But my friend wasn't old. He was about my age. He had a trendy, digital job.

The funeral is done, the grief lingers on. What should I do now?

Do I stop following his accounts?

I don't want to. In part, because I don't want to think he's gone. But on a practical level, if I don't then I know some well-meaning AI will suddenly say "Hey! You haven't talked to your mate in aaaaages! Why not give him a poke?! ?"

Of course, if I do unfollow him, I'll get a whole bunch of "people you know also follow..." with his face staring back at me.

Or, perhaps one of the apps he authorised will get hacked, and I'll be left facing post-mortem-spam.

Christian Payne

@Documentally

I just got a direct message from @craigmanganello ..nearly 4 years after he died. Feels like spam has defiled his grave. ❤️ 1💬 5🔁 006:40 - Sat 16 June 2012

Should I delete his number from my phone?

Last year a different friend gave up on social media and deleted all her accounts. One day, an app in my pocket buzzed to tall me that she'd joined their service. I was flabbergasted and sent her a quick hello.

It wasn't her. She'd changed phone numbers and - as happens in the UK - the number was eventually assigned to someone else. It was that interloper who'd joined. But the social-network-graph doesn't really account for such changes to "unique" identifiers.

Deleting someone from your contacts has a grim feeling of finality to it. Perhaps I just need to get it over and done with.

How do you let people know?

People don't have little paper address books any more. The ones they do have are locked behind the unforgiving fortress of biometric security. So his family asked friends to reach out to more nth-degree-of-separation friends to let the know the dreadful news and details of the funeral.

I remember my parents making such phone calls when I was a kid. Always the same patter, "Are you sat down? I'm afraid I have some horrible news..."

But the enforced brevity of Twitter doesn't really allow for such niceties, does it?

Sending an email seems... I don't know... impersonal? But there's a whole bunch of mutual friends who exist, for me, only as text on a screen. Do I call them? Do I BCC them? What emoji conveys the right amount of reverence for the situation?

How, in a little screen, do you mentally prepare someone for a shock? Is the subject line "Some terrible news about our friend" too much? Too little?

The Funeral

I would live-tweet a funeral, take selfies with the deceased

Tacky - "Weird Al" Yankovic

I mean, that's crazy, right? But that's how a bunch of us knew him - through social media.

Yes, selfies at a funeral are a thing - they're a social object which has been the subject of much discussion. Some friends of mine advocate photographing the funeral in the same way you would any other significant occasion.

What's the right balance? Checking in on FourSquare seems a little disrespectful, but is it OK to live-stream the ceremony on Periscope to those who can't make it?

Preparing for the inevitable

Perhaps selfishly, my thoughts turned to my own mortality. I've got life assurance to take care of my family financially, and I have a Will to sort out any property issues - but what about my digital life?

Google has the euphemistically named "Inactive Account Manager". I've set it so my family has access to my account should I die.

Facebook has a "Legacy Contact setting".

A legacy contact is someone who you choose to manage your account after you pass away. They'll be able to do things like pin a post on your Timeline, respond to new friend requests and update your profile picture. They won't post as you or see your messages

My password manager has a sharing function which, if I set it correctly, should smooth access to some of my accounts.

And as for everything else...? Who knows. Do the half-a-dozen people who follow me on an obscure social network care that I'm gone? Will it be distressing for my relatives to receive reminder emails from them?

The aftermath

Every so often, someone - who hadn't realised he's gone - drops a note on his wall wishing him happy birthday. Or sends an @ message to a few of us. It's not their fault, but it is hard not to feel a little pang of anger and then sadness.

The real pain is looking through the list of friends streaming by and wondering if I'm being good enough to them.

Is "Liking" a Tweet a substitute for a pint of a beer and a natter? Is scrawling "Happy birthday" in a WhatsApp message as good as ringing up for a chat?

And now?

If you'd like to donate to my friend's memorial, there is a JustGiving campaign running (oh, look, another social network).

Suicide is the single biggest killer of men aged under 45 in the UK.

If you're in pain or distress, you're not alone. You can always find someone to talk to.

#death #NaBloPoMo

Death Etiquette For The Social Media Age

My friend died last month. When I was a lad, it seemed that I was always going to funerals for some ancient relative. Everyone looks old when you're a kid. But my friend wasn't old. He was about my age. He had a trendy, digital job. The funeral is done, the grief lingers on. What should I do now? Do I stop following his accounts? I don't want to. In part, because I don't want to think…

Terence Eden’s Blog

Promoting With 💩

https://shkspr.mobi/blog/2016/11/promoting-with-%f0%9f%92%a9/

One of the hardest things about attending conferences with multiple tracks is deciding which talk to go to.

👩🏻‍💻Karina Pencarski

@kary_key

Replying to @hackferencebrum@hackferencebrum is breaking my heart, 2 amazing talks by @supersole with Servo and @etiene_d with my lovely Lua at the same time! #hf2016 ❤️ 2💬 1🔁 013:21 - Fri 21 October 2016

If your friends are talking opposite each other, it is particularly hard.

As a speaker, though, it's even tougher to know you're up against a brilliant speaker. Speaking is hard - speaking to an empty room is... painful.

So, I thought I'd try a little experiment at the recent Hackference Brum. I was giving a talk about Unicode which - let's face it - isn't the most glamorous subject in the world. But, to most people, Unicode == Emoji.

And what's everyone's favourite Emoji?

Yup - I purchased a Big Plushie ?!

Touting your talk at a conference always runs the risk of looking desperate - or simply being awkward. So I decided to ask people if they'd like to take a selfie with my poo. Turns out, people really like that idea!

Terence Eden is on Mastodon

@edent

And @sammachin gets the first sweet at #hf2016! pic.x.com/bkl7bzvfpa

❤️ 3💬 0🔁 008:05 - Fri 21 October 2016

Paul Cooper

@pgc

Don't be fooled - that's not an emoji it's @edent #HF2016 pic.x.com/4x3q0bbtlf

❤️ 1💬 0🔁 008:35 - Fri 21 October 2016

Terence Eden is on Mastodon

@edent

Who loves 💩 selfies?
EVERYONE!
#HF2016 pic.x.com/be7gc78h98

❤️ 4💬 0🔁 009:41 - Fri 21 October 2016

Montpellier Integrated

@MontPRCreative

It looks like @guillou_gaelle and @martinthorne are having an excellent #FriYAY at #HF2016 pic.x.com/as8o1kbayy

❤️ 2💬 0🔁 010:01 - Fri 21 October 2016

Samathy Barratt

@Samathy_Barratt

.@hackferencebrum @edent @lexicobob 💩 Emoji!!💩 pic.x.com/u03o7osyvd

❤️ 2💬 0🔁 009:39 - Fri 21 October 2016

Terence Eden is on Mastodon

@edent

All the ladies love...
#hf2016 pic.x.com/qgtsloaus2

❤️ 4💬 0🔁 008:19 - Fri 21 October 2016

Andrew Faraday

@MarmiteJunction

I selfied myself photobombing a poo, photobombing @edent's talk at #hf2016
#hackfurnace pic.x.com/5u6hedqqhf

❤️ 0💬 1🔁 015:18 - Fri 21 October 2016

flaki

@slsoftworks

Me & @edent looking poo-tiful on our combo-selfie with Mr 💩
💁✨ #hf2016 pic.x.com/waka6jwxog

❤️ 1💬 0🔁 013:51 - Fri 21 October 2016

Terence Eden is on Mastodon

@edent

Looks like @JakeLPrice has made a new friend at @hackferencebrum!
#hf2016 pic.x.com/zqt4rgjw0z

❤️ 1💬 0🔁 012:20 - Fri 21 October 2016

And, with each selfie, I got a chance to talk to people and let them know about my talk. It was a fun way to chat and - thankfully - seemed to pay off!

Loads of people turned up, they asked brilliant questions at the end, and seemed to really dig my talk.

HackSoc Nottingham

@hacksocnotts

In the beginning there was nothing... Unicode lols with @edent @hackferencebrum #hf2016 pic.x.com/efdy2u9ns0

❤️ 2💬 0🔁 015:26 - Fri 21 October 2016

Hackference Brum

@hackferencebrum

🤣 @edent sharing how the hell to deal with Unicode and getting new characters in at #hf2016 pic.x.com/xmxnvt7uou

❤️ 1💬 0🔁 015:44 - Fri 21 October 2016

be7.is

@bevishalperry

.@edent talking the pitfalls, history and future of Unicode and how everyone can improve it. @hackferencebrum #hf2016 pic.x.com/hfkap5hqzi

❤️ 1💬 0🔁 015:33 - Fri 21 October 2016

So, there we go - a fun little experiment which seemed to pay off. I got to meet so many fun new people and made them take some highly questionable photos ?

It was so much fun, that I did it all again when I went to MozFest!

#hackference #hf2016 #NaBloPoMo #presentations

Promoting With 💩

One of the hardest things about attending conferences with multiple tracks is deciding which talk to go to. 👩🏻‍💻Karina Pencarski@kary_keyReplying to @hackferencebrum@hackferencebrum is breaking my heart, 2 amazing talks by @supersole with Servo and @etiene_d with my lovely Lua at the same time! #hf2016❤️ 2💬 1🔁 013:21 - Fri 21 October 2016 If your friends are talking opposite each other, it is pa…

Terence Eden’s Blog

Book Review: Seeing Like A State - James C. Scott

https://shkspr.mobi/blog/2021/11/book-review-seeing-like-a-state-james-c-scott/

In this wide-ranging and original book, James C. Scott analyses failed cases of large-scale authoritarian plans in a variety of fields. Centrally managed social plans misfire, Scott argues, when they impose schematic visions that do violence to complex interdependencies that are not—and cannot—be fully understood. Further, the success of designs for social organization depends upon the recognition that local, practical knowledge is as important as formal, epistemic knowledge. The author builds a persuasive case against “development theory” and imperialistic state planning that disregards the values, desires, and objections of its subjects. He identifies and discusses four conditions common to all planning disasters: administrative ordering of nature and society by the state; a “high-modernist ideology” that places confidence in the ability of science to improve every aspect of human life; a willingness to use authoritarian state power to effect large- scale interventions; and a prostrate civil society that cannot effectively resist such plans.

I've lost count of the number of people who have recommended that I read this book. It dismantles some of the arguments for centralised authority and remaking the world to satisfy the administrative demands of the state.

Is this a call for anarchism? No. I think it's an early plea for more user centred design.

It's really easy for people to design a system that works for people like them. It's much harder to design something which works with "messy" reality.

The first few chapters were highly relevant to my work. It's hard to administer reality - but forcing people / land / systems to conform to something uniform rarely works in practice.

I found the chapters on High-Modernist stuff bit weird. I get the relevance of physical architecture and its parallels to computer architecture, but I felt it went a bit too deep into the weeds of Soviet agricultural plans for my liking. I ended up skim reading some of the chapters on farming. That said, it has a good discussion of colonialism and the perils of thinking you know better than people with local knowledge.

It does an excellent job of explaining the limits of technocracy. There's even some fome feminist critique of centralised design, which is good to see.

Ultimately, it sold me on the idea that standards only works when everyone gets a say in how they are developed. And that they need to be flexible enough to cope with the present and future reality. If we want people to embrace change, it can only be via radical participatory democracy.

A sobering and vitally important book. Highly recommended for anyone who is involved in policy design or standardisation.

Here are some of the choice excerpts I found interesting:

Terence Eden is on Mastodon

@edent

Replying to @edent"There is, then, no single, all-purpose, correct answer to a question implying measurement unless we specify the relevant local concerns that give rise to the question." ❤️ 5💬 0🔁 117:57 - Mon 25 October 2021

Terence Eden is on Mastodon

@edent

Replying to @edentRelevant to yesterday's discussion on storing names.
And also looking back to the Google / Facebook "real names" movement.
Names are socially flexible. Can they ever really be captured in a database? pic.x.com/NFEyvqXFKB

❤️ 3💬 1🔁 020:09 - Tue 26 October 2021

Terence Eden is on Mastodon

@edent

Replying to @edentThis is so true it hurts.

How many times have you been through a re-org which shuffled people around on PowerPoint decks with little regard for what they want, how they are incentivised, or how they work? pic.x.com/ez4dj6yQIm

❤️ 21💬 6🔁 020:47 - Mon 01 November 2021

Terence Eden is on Mastodon

@edent

Replying to @edentThis book - Seeing Like A State - is doing its best to make me question my professional career! pic.x.com/G2mMaaIu2f

❤️ 3💬 0🔁 107:35 - Wed 03 November 2021

Terence Eden is on Mastodon

@edent

Replying to @edent*nodding intensifies* pic.x.com/yy1if0M9Ih

❤️ 4💬 0🔁 008:08 - Thu 04 November 2021

#BookReview #NaBloPoMo

Book Review: Seeing Like A State - James C. Scott

In this wide-ranging and original book, James C. Scott analyses failed cases of large-scale authoritarian plans in a variety of fields. Centrally managed social plans misfire, Scott argues, when they impose schematic visions that do violence to complex interdependencies that are not—and cannot—be fully understood. Further, the success of designs for social organization depends upon the rec…

Terence Eden’s Blog

A Complete List of Every UK Government Domain Name

https://shkspr.mobi/blog/2015/11/a-complete-list-of-every-uk-government-domain-name/

Eight years after I published this blog post, I helped officially release all these domain names as open data! Funny how life works out, eh?

Would you like to know every domain name the UK Government had registered? Of course you would! There could be all sorts of interesting tit-bits hidden in there (ProtectAndSurvive.gov.uk? EbolaOutbreak2017.nhs.uk? MinistryOfTruth.police.uk?)

Rather than relying on Freedom of Information requests, or Open Data, we can go straight to the source of domain names - the DNS!

Shut Up And Give Me The Codez!

Download all UK Government host names.gov.uk 15,436 records.nhs.uk 4,877 records.police.uk 466 records.mod.uk 268 records.parliament.uk 91 records

That's... quite a lot! The majority are host names - only around 2,247 of the GOV.UK ones are domain names. Many of them are not currently live.

Still, I wonder how many are new?

Steph Gray

@lesteph

Not intended snarkily, but has web rationalisation/no new govt domains been formally abandoned as a policy now? ❤️ 0💬 1🔁 013:22 - Mon 09 November 2015

Steph Gray

@lesteph

Replying to @charlottejee@charlottejee stuff like exportingisgreat.gov.uk, workplacepensions.gov.uk etc ❤️ 0💬 0🔁 013:23 - Mon 09 November 2015

Charlotte Jee

@charlottejee

Replying to @lesteph@lesteph This seems to say new domains need approval rather than that they aren't allowed: gov.uk/government/pub… ❤️ 1💬 1🔁 013:26 - Mon 09 November 2015

The Gov.UK file is a CSV which also show when the domain was first registered (if available).

Geeky Details

The Domain Name System (DNS) lists every single domain name (example.com). It tells your computer which IP Address is associated with a Domain Name. If your local DNS doesn't know where example.gov.uk lives, it goes to the ISP's DNS. If they don't know, they ask an upstream provider's DNS. And so on, until someone asks the .gov.uk nameserver for an authoritative response.

So, can you download every domain name in existence? No, not easily. It usually involves filling out lots of forms and giving some compelling reason why you want it.

However, Rapid7's sonar project provides a sort of "best guess" for all the domain names which it can see.

To download the entire file is 12GB. That's the zipped version.

Once unzipped, it's a whopping 67GB

A quick look at the file shows it contains 1,408,097,159 records. Youch! That's a lot of domain names!

This is what the file looks like

$ head 20150926_dnsrecords_allcshengmei.com.h310.6dns.net,a,103.225.196.101reseauocoz.cluster007.ovh.net,cname,cluster007.ovh.netcse-web-cl.comunique-se.com.br,a,200.166.77.69ext-cust.squarespace.com,a,198.185.159.176ext-cust.squarespace.com,a,198.185.159.177ext-cust.squarespace.com,a,198.49.23.176ext-cust.squarespace.com,a,198.49.23.177ghs.googlehosted.com,cname,googlehosted.l.googleusercontent.comisutility.web9.hubspot.com,cname,a1049.b.akamai.netsendv54sxu8f12g.ihance.net,a,54.241.8.193sites.smarsh.io,a,199.47.168.63www.triblocal.com.s3-website-us-east-1.amazonaws.com,cname,s3-website-us-east-1.amazonaws.com*.01ete21.cn.cname.yunjiasu-cdn.net,a,162.159.210.34*.01ete21.cn.cname.yunjiasu-cdn.net,a,162.159.211.34

As a brief primer, a CNAME points to another domain name. An A Record points to an IP address. There are lots of different domain records.

Ok, so let's get all the *.gov.uk records out of there...

grep "gov\.uk" 20150926_dnsrecords_all0-19insalford.info,soa,ns0.ictservices.co.uk postmaster.salford.gov.uk 2010022204 28800 7200 604800 86400019186.gov.ukpfl.cn,a,122.9.230.117100days.local.gov.uk,a,198.154.241.231101.gov.uk,a,216.146.46.10101.gov.uk,a,216.146.46.11101.gov.uk,mx,20 sms2.101.gov.uk101.gov.uk,ns,ns1.p08.dynect.net

Ah! Ok, we're picking up some websites which are pointing to a gov.uk site (potentially useful) and some false positives like "019186.gov.ukpfl.cn". Let's just look at records where the first column ends with .gov.uk":

grep "\.gov\.uk," 20150926_dnsrecords_all100days.local.gov.uk,a,198.154.241.231101.gov.uk,a,216.146.46.10101.gov.uk,a,216.146.46.11101.gov.uk,mx,20 sms2.101.gov.uk101.gov.uk,ns,ns1.p08.dynect.net101.gov.uk,ns,ns2.p08.dynect.net101.gov.uk,ns,ns3.p08.dynect.net101.gov.uk,soa,ns1.p08.dynect.net hostmaster.cscdns.net 2014121100 3600 600 604800 18001901redirect.nationalarchives.gov.uk,a,193.132.104.1511sttouch.powys.gov.uk,a,212.219.229.791t6c3c0p2r0m934.forestry.gov.uk,a,212.38.180.452011.census.gov.uk,a,94.126.106.1322014.colneyheathparishcouncil.gov.uk,a,81.27.85.112050-calculator-tool-wiki.decc.gov.uk,cname,wiki.2050.org.uk

OK, so how do we de-duplicate these? The first thing to do is manipulate the data. We only want the first column. There are an number of ways to do this in Linux, I prefer to use the Python tool CSVfilter.

To install sudo pip install csvfilter.

To grab only the first (zeroth) column
cat 20150926_dnsrecords_all | csvfilter -f 0 > out.csv

Now, this doesn't quite work. Why? Because some DNS records contain incredibly strange data! You can manually clean up the data, but that's a bit boring and utterly impossible to load into Excel or any other normal editor.

Here's what I did...

  • Copy all the lines containing gov.uk into a new file
    grep "\.gov\.uk," 20150926_dnsrecords_all > govuk.csv
  • Create a new file with only the first column
    cat govuk.csv | csvfilter -f 0 > govuk0.csv
  • Sort the file and make sure each line in unique
    sort govuk0.csv | uniq > govuk.txt
  • Hey presto! A more-or-less complete list of every .gov.uk website which is registered. The same can be performed for .NHS.uk, .police.uk, .MOD.uk etc.

    Getting The Dates

    Time to crack out the Ruby!

    Using the WHOIS library, I wrote a simple script to parse the text records and query when the domain name was created.

    #!/usr/bin/env rubyrequire 'whois'c = Whois::Client.newFile.open( "govuk.txt" ).each do |line| begin r = c.lookup(line.chomp) puts "#{line.chomp},#{r.created_on}" rescue Whois::Error => e rescue StandardError => e endend

    This isn't perfect - there are only records for the third level of gov.uk - and no records at all for Parliament, MOD, Police, and NHS. It is also a bit slow to run through the thousands of records - but we can see a few interesting bits and bobs.

    Created in 2015

    I suspect some of these are merely renewals, rather than brand new domains.

    seemis.gov.uk,2015-10-29 00:00:00 +0000yjb.gov.uk,2015-10-28 00:00:00 +0000crbonline.gov.uk,2015-10-23 00:00:00 +0100coi.gov.uk,2015-10-14 00:00:00 +0100gibraltar.gov.uk,2015-07-29 00:00:00 +0100dorsetforyou.gov.uk,2015-03-19 00:00:00 +0000ico.gov.uk,2015-03-19 00:00:00 +0000bridgnorthtowncouncil.gov.uk,2015-01-29 00:00:00 +0000

    Oldest

    wdc.gov.uk,2003-06-03 00:00:00 +0100west-dunbarton.gov.uk,2003-06-03 00:00:00 +0100clacks.gov.uk,2003-06-02 00:00:00 +0100bassetlaw.gov.uk,2003-04-29 00:00:00 +0100dti.gov.uk,2003-03-13 00:00:00 +0000

    Sadly, clacks.gov.uk has very little to do with Terry Pratchett!

    That's all folks!

    Spotted anything unusual? Found a better way to do things? Stick a comment in the box!

    If you've enjoyed this post, you can buy me something from my Amazon Wishlist.

    #data #govUk #NaBloPoMo

    A Complete List of Every UK Government Domain Name

    Eight years after I published this blog post, I helped officially release all these domain names as open data! Funny how life works out, eh? Would you like to know every domain name the UK Government had registered? Of course you would! There could be all sorts of interesting tit-bits hidden in there (ProtectAndSurvive.gov.uk? EbolaOutbreak2017.nhs.uk? MinistryOfTruth.police.uk?) Rather than…

    Terence Eden’s Blog