Ruby-Elf and collision detection improvements

While the main use of Ruby-Elf for me lately has been quite different – for instance with the advent of elfgrep or helping verifying LFS support – the original reason that brought me to write that parser was finding symbol collisions (that’s almost four years ago… wow!).

And symbol collisions are indeed still a problem, and as I wrote recently they don’t get very easy on the upstream developers’ eyes, as they are mostly an indication of possible aleatory problems in the future.

At any rate, the original script ran overnight, generated a huge amount of database, and then required more time to produce a readable output, all of which happened using an unbearable amount of RAM. Between the ability to run it on a much more powerful box, and the work done to refine it, it can currently scan Yamato’s host system in … 12 minutes.

The latest set of change that replaced the “one or two hours” execution time with the current “about ten minutes” (for the harvesting part, there are two more minutes required for the analysis) was part of my big rewrite of the script so that it used the same common class interfaces as the commands that are installed to be used with the gem as well. In this situation, albeit keeping the current single-threaded (more on that in a moment), each file analysed consists of three calls to the PostgreSQL backend, rather than being something in the ballpark of 5 plus one per symbol, and this makes it quite faster.

To achieve this I first of all limited the round-trips between Ruby and PostgreSQL when deciding whether a file (or a symbol) has been already added or not. In the previous iteration I was already optimising this a bit by using prepared statements (that seemed slightly faster than direct queries), but they didn’t allow me to embed the logic into them, so I had a number of select and insert statements depending on the results of those, which was bad not only because each selection would require converting data types twice (from PostgreSQL representation to C, then from that to Ruby), but also because it required to call into the database each time.

So I decided to bite the bullet and, even though I know it makes it a bunch of spaghetti code, I’ve moved part of the logic in PostgreSQL through stored procedures. Long live PL/SQL.

Also, to make it more solid in respect to parsing error on single object files, rather than queuing all the queries and then commit them in one big single transaction, I create single transactions to commit all the symbols of an object, as well as when creating the indexes. This allows me to skip over objects altogether if they are broken, without stopping the whole harvesting process.

Even after introducing the transaction on symbols harvesting, I found it much faster to run a single statement through PostgreSQL in a transaction, with all the symbols; since I cannot simply run a single INSERT INTO with multiple values (because I might hit an unique constrain, when the symbols are part of a “multiple implementations” object), at least I call the same stored procedure multiple times within the same statement. This had tremendous effect, even though the database is accessed through Unix sockets!

Since the harvest process now takes so little time to complete, compared to what it did before, I also dropped the split between harvest and analysis: analyse.rb is gone, merged into the harvest.rb script for which I have to write a man page, sooner or later, and get installed properly as an available tool rather than an external one.

Now, as I said before, this script is still single-threaded; on the other hand, all the other tools are “properly multithreaded”, in the sense that their code fires up a new Ruby thread per each file to analyse and the results are synchronised not to step on each other’s feet. You might know already that, at least for what concerns Ruby 1.8, threading is not really implemented and green threads are used instead, which means there is no real advantage in using them; that’s definitely true. On the other hand, on Ruby 1.9, even though the pure-Ruby nature of Ruby-Elf makes the GIL a main obstacle, threading would improve the situation by simply allowing threads to analyse more files while the pg backend gem would send the data over to PostgreSQL (which would probably also be helped by the “big” transactions sent right now). But what about the other tools that don’t use external extensions at all?

Well, threading elfgrep or cowstats is not really any advantage on the “usual” Ruby versions (MRI18 and 1.9), but it provides a huge advantage when running them with JRuby, as that implementation has real threads, it can scan multiple files at once (both when using asynchronous listing of input files with the standard input stream, and when providing all of them in one single sweep), and then only synchronise to output the results. This of course makes it a bit more tricky to be sure that everything is being executed properly, but in general makes the tools just the more sweet. Too bad that I can’t use JRuby right now for harvest.rb, as the pg gem I’m using is not available for JRuby, I’d have to rewrite the code to use JDBC instead.

Speaking about options passing, I’ve been removing some features I originally implemented; in the original implementation, the arguments parsing was asynchronous and incremental, without limits to recursion; this meant that you could provide a list of files preceded by the at-symbol as the standard input of the process, and each of that would be scanned for… the same content. This could have been bad already for the possible loops, but it also had a few more problems, among which there was the lack of a way to add a predefined list of targets if none was passed (which I needed for harvest.rb to behave more or less like before). I’ve since rewritten the targets’ parsing code to only work with a single-depth search, and relying on asynchronous arguments passing only through the standard input, which is only used when no arguments are given, either on command line or by default of the script. It’s also much faster this way.

For today I guess all these notes about Ruby-Elf would be enough; on the other hand, in the next days I hope to provide some more details about the information the script is providing me.. they aren’t exactly funny, and they aren’t exactly the kind of things you wanted to know about your system. But I guess this is a story for another day.

"After nearly two decades, a dazzling Nanyang Technological University building will finally get a facelift to make it safe for birds.
[...]
Since the four-storey building’s completion in 2006, its large glass surfaces and funnel-like shape have frequently injured or killed birds, Project Avigate has found."

https://www.straitstimes.com/singapore/environment/ntu-students-successfully-fund-raise-to-make-school-building-safe-for-birds

#Singapore #Buildings #Birds #Collisions

I don't disbelieve these govt #statistics on reported fatalities from road traffic #collisions in Great #Britain where #drugs were involved (nor the quote from every constabulary that its increased by 70% since 2014), but what I find strange is I've been monitoring news reports of these myself for the same period and haven't seen /that/ many - I suspect this means a lot of incidents aren't even making the news - maybe middle class families are somehow managing to keep these incidents out of the media, or that there are simply more RTCs occurring in general than the news media are able to report on?

https://www.gov.uk/government/statistics/developing-drug-driving-statistics/drugs-in-reported-road-fatalities-in-great-britain-data-to-2023-summary--2

#Development #Pitfalls
Oops, my UUIDs collided! · How Universally Unique Identifiers can collide https://ilo.im/1682w2

_____
#Identifiers #Collisions #UUID #Mistakes #Attacks #Hashes #JavaScript #WebDev #Frontend #Backend

Sigh....

"A bus stop in Dairy Farm Road, which is located near Singapore’s green “heart”, recently claimed several avian victims as multiple birds were seen flying into its clear panels.

Collisions like this could happen as birds may not be able to perceive the clear panels as barriers, experts say."

https://www.straitstimes.com/singapore/environment/at-least-five-birds-spotted-crashing-into-clear-panels-at-dairy-farm-bus-stop-within-24-hours

#Singapore #Birds #Collisions

The thread about runaway tramcars; what happens when the brakes won’t work?

On this day in 1918 (June 23rd when written) there was an “Extraordinary Tramway Incident” in Edinburgh as a result of which, miraculously, nobody was hurt. This was the days of cable traction, when the city’s public transport wound its way slowly and noisily around the streets hauled by an endless loop of moving cable beneath the setts.

It was just after 10 O’clock in the morning and an empty car (they were always called cars, never trams) was standing awaiting its next service at the Braids terminus, near where Comiston Road meets Braid Hills Road. The driver and conductor were on their break in the adjacent shelter (a replacement version of which is still there to this day). But the vehicle’s mechanical brakes had not been fully set and, imperceptibly slowly at first, the car began to creep away down Comiston Road towards the City.

The driver and conductor gave chase as soon as they noticed but were already too late and were unable to catch it as it began to speed up. The cable which moved the tramcars when in service was attached to by means of a mechanical “gripper” and whizzed anything attached to it along at a rather sedate 9½ mph (to which it was limited by Board of Trade regulations). The downhill runaway quickly passed this limiting speed and quickly caught up with the car running in service ahead of it. Fortunately there were no passengers aboard. The conductress saw the approaching danger, called a warning to the driver, and sensibly jumped for it. The inevitable collision happened and the two cars became entangled together. Luckily neither derailed. Less fortunately the driver – who had heroically stayed at his post – found that the gripper which connected his car to the traction cable had become jammed; the conjoined wreckage was thus firmly attached to the cable and was being hauled inextricably towards busy Morningside at a slightly less than terrifying nine and a half miles per hour. The danger was still real however as this was where Comiston Road changed from rural to dense urban in nature and the driver now had no way to stop at any approaching junctions, for any other tram cars ahead of him or to slow for any obstructions such as pedestrians, cyclists, horses, children crossing or workmen carrying sheets of plate glass across his path like they did in the movies then.

At Belhaven Terrace, just one mile from where the runaway had started, there was a set of points manned by a pointsman for switching the cars which had terminated at Morningside Station between tracks. The driver managed to call out his predicament to the pointsman as he sailed helplessly past. The pointsman’s hut had a telephone connection to the winding house at Tollcross which was powering the cable and after a brief call the winding engine was stopped. This brought the runaway – and half of the tramway network in the south and west of the city which shared that cable – safely to a halt. It took time to untangle the damaged cars and extricate the gripper from the cable but within an hour and a quarter from the start of the incident the network was back up and running again as if nothing had ever happened. Sadly no further details were reported in the papers

Runaways tramcars were fortunately very rare, but not unheard of. In April 1890 a horse-drawn vehicle had ran away down Montrose Terrace in the east of the city. One of the animals fell but its panicked companion dragged it and the tramcar along the ground a further 1,000ft before coming to rest outside the Abbey Church. “The passengers, who were greatly alarmed, were not, however, injured”. The same cannot be said about the poor fallen horse, as “on examination it was found… [to] not likely be of any more use.” Sadly it was probably a one way ticket to Cox’s Glueworks in Gorgie for that victim.

On September 30th 1909 a cable tramcar was involved in a potentially much more deadly accident at Waterloo Place. The car approached the terminus from the direction of Abbeyhill to pick up passengers from a large crowd intending to travel to Musselburgh for the races, but failed to stop in the right place. The driver brought it to a halt further on at the top of Leith Street – outside the General Post Office – and got out to inspect why it had failed to stop in the correct place. The curious and frustrated crowd naturally gravitated towards the vehicle to see what the problem was and if they could board it.

Perhaps the driver prodded around underneath too hard, or perhaps an onlooker knocked something they ought not to have, but in an instant the car suddenly lurched back in the direction of Waterloo Place – and the thronging crowd – seemingly attached to the return cable by forces unknown. The vehicle ploughed through the crowd resulting in six people (including three police men) being injured. Again it was lucky that cable haulage did not work at faster speeds. But all was not yet over; “just at that moment a motor car.. appeared on the scene“. The motor became entangled in the front of the tramcar and was dragged along the road by it for thirty yards. The driver jumped clear, narrowly avoiding behind run over by another tramcar coming up Waterloo Place from Regent Road. And then, as soon as it started, the excitement was over: the tramcar released its unexpected grip on the traction cable and the tangle of public and private transportation ground to a halt.

Of the four hospitalised, three had been in the car; the driver Archibald Carmichael and his passengers John McArthy and James Paton, through for the day from Gourock and Port Glasgow for the races. The fourth was a pedestrian, Mrs Mathieson of Gayfield Square. This wasn’t the result of black magic or enchantment however as a simple explanation was soon found. As the tramcar had approached Regent Road it had to switch between traction cables by releasing one cable with the rear gripper and grabbing the other with the front gripper (each tramcar had a front and rear gripper to attach itself to the cables.) At it had moved across the junction where Montrose Terrace branched off London Road the gripper had damaged the cable running in the slot between the rails. This had cut into some of the wire strands which had came started to unravel, preventing the gripper from releasing properly when the driver tried to stop on reached Waterloo Place. The fraying cable meantime had run around the pulley and come back towards the now stationary tramcar from the opposite direction and when the tangle of loose strands passed through the released gripper they quickly became tangled in it and enough unravelling wire built up to suddenly start the car in motion again in the return direction.

It would prove to be a bad day for the Edinburgh & District Tramways Co. and there had already been an accident earlier that morning when at 11:40AM a car overshot a set of points and collided with the pawl near the Abbey Church between Abbeyhill and Meadowbank. This was an emergency stop device mounted in the traction cable slot beneath the road surface which, if it was hit by a tramcar gripper, detached it from the traction cable and brought the tram to a very instant stop. The pawl was a violent but important measure which prevent the gripper from entering and becoming entangled in the large underground pulleys around which the cable ran. The shock of hitting the pawl in this case was severe enough to snap the car’s front axle. But these were different and more efficient times; the tramway depot at Shrubhill simply sent out a gang with a complete new bogey, which they swapped out at the side of the road in Abbeyhill before getting the car quickly back in service.

The return to service lasted just 15 minutes following the accident before the network had to be stopped again! A tramcar going from the Bridges to Leith Street had collided with another motor car and once again the pair became entangled and took some time to clear. This was the third and final accident of the day.

An altogether more tragic runaway accident took place on Saturday 17th October 1925 when an electric tramcar proceeding down a single line section on Ardmillan Terrace ran away and jumped the points at the foot of the hill, crashing into St. Martin’s Episcopal Church. Young Edward Stirling, aged just 8, had been out to buy a toy train for his brother who was in hospital. He was killed instantly when struck by the tram and was buried in Saughton Cemetery on Wednesday 21st. The tram driver, 3 passengers and another pedestrian were hospitalised. The Corporation sent a wreath and his funeral was attended by Councillor Mancor, Convenor of the Tramway Committee and R. S. Pilcher, the general manager of Corporation Tramways. The Lord and Lady Provost sent a letter of sympathy. A subsequent Board of Trade enquiry found no fault with the tramway equipment or tramcar and did not apportion blame to the driver, who had tried to use the resistance brake to slow the tram followed by the hand brake. There was some popular discontent with the Tramways, a correspondent by the pen name of A. Mother wrote to the ‘News to complain about the speed of the electric trams – “too fast for either safety or comfort“- and to protest about plans to increase their speed even further.

The next runaway took place in spectacular fashion on Saturday June 1st 1929. Miraculously, nobody was injured when car No. 349, waiting at the Liberton terminus with no crew but four elderly passengers aboard ran off down Liberton Brae. Anyone who has ever tried to cycle up (and down) that road can attest just how severe the gradient quickly gets! As the crew availed themselves of the facilities in the terminus shelter they didn’t notice their vehicle slowly begin to roll off down the hill; they had not applied the mechanical hand brake and the pressure had leaked out of the air brake hose causing it to slowly release.

The four passengers (aged 59, 71, 71 and 84) were subjected to a terrifying half mile ride down the Brae before No. 349 came to the corner at Alnwickhill Road, jumped the tracks, slid across the road and pavement and impaled itself on a pole for the overhead wires before caming to rest in the garden of 42 Liberton Brae.

Quite how only one passenger suffered only light bruising and all four had walked away from this defies logic. Indeed the two 71 year olds who had been aboard, a married couple from Leith Walk, simply waited for the next car and carried on home as if nothing else had happened. The house at number 42 was similarly unscathed, although the same could not be said for its garden wall, gate and neat privet hedging.

As a result of this incident protective barriers were installed on the corner outside numbers 42 to 46 where they remain to this day; the dents in their metalwork show they still serve their intended purpose well.

Six years later, in 1935, an accident took place only a few hundred metres down the Brae at Braefoot Terrace. The current collection pole of a tramcar proceeding uphill became dislodged from the current wire and the vehicle lost power and ground quickly to a halt. The trailing SMT bus was following too closely to stop in time and swerved off the road to avoid a collision. Instead it demolished the shopfronts of a James Baxter’s butchers, Adam Smith’s chemist and the branch of the Commercial Bank of Scotland. There were fortunately no major injuries; the butcher’s boy had a lucky escape as he had just been sent outside to clean the windows only to find a thirty-two seater bus baring down on him. He was able to jump clear in the nick of time.

On 25th October 1945, 64 year old Brownlow Grigor of Leith, a Corporation tram driver with 30 years experience was fined £3 by the Burgh Court for “having driven a tram culpably and recklessly“. Grigor was in charge of car No. 42 and had not been paying sufficient attention as he was trying to stow away his thermos flask. He had approached the sharp bend where cars travelling between Morningside and Marchmont turned off Church Hill and onto Greenside Gardens too fast and the laws of physics did the rest. His vehicle jumped the tracks and demolished a 24-foot section of the wall of – coincidentally – number 42 Greenhill Gardens. The embarrassment was all the more severe as this house was St. Bennets, the official residence and private chapel of the Roman Catholic Archbishop of Edinburgh and St. Andrews! Grigor’s claim that it was defective rails that were at fault was not upheld.

Our last calamity took place in 1951 and it was the heroic figure of James Ferguson, a 53 year old worker from Portobello Power Station, who averted a real catastrophe. The busy tramcar he was on was hit by a lorry near Piershill which injured the driver and caused him to lose consciousness. Ferguson leapt to his feet, ran the length of the car, punched through the glass door to the driver’s compartment with his bare fist and let himself into the cab to disengage the power lever and apply the brakes. Ferguson was treated in hospital for cuts to his hands but was otherwise unscathed.

https://www.flickr.com/photos/127340508@N05/26917130321/in/photolist-226kxHm-2nsTewY-2nTHjmS-2hfcDKE-2iU8A9N-2jDXFBt-2n4mPiw-2n5L7wp-2j2SqzQ-QWjur3-UCFFfN-D5VnNV-dZN3tD-2iriC1Q-2eqrcdo-JXr4pn-DmKy8a-awA1sk-nwjsJs-awACNn-SETEZM-86nFZc-XaX2By-n1khke-2gwCUbu-QMWmmF-SYsgsg-KR12f3-e8bKFa-DwCURs-H1zk4P-E5T5Gs-2oCJnjm

If you have found this useful, informative or amusing, perhaps you would like to help contribute towards the running costs of this site – including keeping it ad-free and my book-buying budget to find further stories to bring you – by supporting me on ko-fi. Or please do just share this post on social media or amongst friends.

These threads © 2017-2025, Andy Arthur.

NO AI TRAINING: Any use of the contents of this website to “train” generative artificial intelligence (AI) technologies to generate text is expressly prohibited. The author reserves all rights to license uses of this work for generative AI training and development of machine learning language models.

#Abbeyhill #Accident #Accidents #Ardmillan #CableTrams #Collisions #Edinburgh #HorseTrams #June1 #June23 #Liberton #Suburbs #Tram #Trams #Tramway #Tramways #WaterlooPlace

La Volte - Mathias ECHENAY présente "Collisions par temps calme" de Stéphane BEAUVERGER

https://video.lavolte.net/w/kduHXTGjLqdGnyxYUG43iK