Vehicle Automation: Moved from Charlie Alliston Thread

[jefmcg]

There is a suggestion that the somewhat overstated response to this incident in the media may in someway be part of a bid by a lobby group funded by the motoring industry to remove cyclists from the roads in order to smooth the way for self driving cars. This may seem like a conspiracy but it is certainly the case that jaywalking laws in the US were a direct result of a similar campaign against pedestrians (or as I like to call them, people just going about their business.)

That's not my point It's a general discussion and it doesn't belong hidden away more than 1000 posts into a thread that isn't obviously relevant.

 

[theclaud]

There is a suggestion that the somewhat overstated response to this incident in the media may in someway be part of a bid by a lobby group funded by the motoring industry to remove cyclists from the roads in order to smooth the way for self driving cars. This may seem like a conspiracy but it is certainly the case that jaywalking laws in the US were a direct result of a similar campaign against pedestrians (or as I like to call them, people just going about their business.)

It's not so much that it's part of a conspiracy but that the moment of heightened consciousness and hysteria over pedestrian/cyclist conflict will inevitably be seized as an opportunity by economic and political interests for whom people in streets are a fundamental inconvenience. The same thing happens when lorry drivers kill cyclists - the operators go into overdrive trying to bully cyclists out of the way, because the alternative would be the cost and inconvenience of not killing people. The positive thing is that in foregrounding this they inevitably expose their own faultlines - for example lorry operators inadvertently demonstrating with all that blind spot and seat 'swap' cobblers that their vehicles are unfit to be on the road. Thank fark, at this moment, that we have a robust national cyclists' organisation operating in the interests of its members. Oh wait - we sold that one up the swannee...

 

[srw]

the operators go into overdrive trying to bully cyclists out of the way, because the alternative would be the cost and inconvenience of not killing people

Well - that one went well, didn't it?

 

[srw]

And if the developer decides that overtaking a cyclist with 10 cm to spare at speed is perfectly acceptable, that's exactly what will happen

You know as well as I do that no single developer is going to decide that sort of thing. For a start, that's not how corporate software development is done, and secondly the problem of dealing with unexpected and unpredictable circumstances is too big to be dealt with without significant machine learning.

Sooner or later we're going to need robot ethicists.

If only there was a profession which had thought deeply about ethics for over 2000 years and had developed a specific model for thinking about this sort of problem over 100 years ago.

There is also the small matter of being able to trust the car manufacturers to implement the rules honestly.

There is. But it's an awful lot easier to catch them out if the rules are being flouted every single day on the open road than if the piss-taking only comes into play in specific extremely controlled circumstances. And it's not as if the car manufacturers haven't been dinged really quite hard.

aywalking laws in the US were a direct result of a similar campaign against pedestrians

It's worth flicking through the wikipedia article on the subject. As that article says, "Jaywalking is illegal in over 10 countries due to the health risks" - and when you look down the actual law you realise that the US is in a minority of one in the absolute priority it actually gives to motorised traffic - and even in the US the law is often not enforced.

 

[Tin Pot]

You know as well as I do that no single developer is going to decide that sort of thing. For a start, that's not how corporate software development is done, and secondly the problem of dealing with unexpected and unpredictable circumstances is too big to be dealt with without significant machine learning.


If only there was a profession which had thought deeply about ethics for over 2000 years and had developed a specific model for thinking about this sort of problem over 100 years ago.


There is. But it's an awful lot easier to catch them out if the rules are being flouted every single day on the open road than if the piss-taking only comes into play in specific extremely controlled circumstances. And it's not as if the car manufacturers haven't been dinged really quite hard.



It's worth flicking through the wikipedia article on the subject. As that article says, "Jaywalking is illegal in over 10 countries due to the health risks" - and when you look down the actual law you realise that the US is in a minority of one in the absolute priority it actually gives to motorised traffic - and even in the US the law is often not enforced.

Hmm, broadly I'm with you.

But as someone who examines the software development lifecycle in corporations, I'm not sure that I would agree. Whilst some decisions are made by teams, or even given to the business, a lot of application logic is determined by whatever the developer thinks they can do. Don't get me started on peer review, testing and segregation of duties.

Can a developer put in a "10cm rule" and do they? Yes, absolutely. Agile and DevOps can make this better or in most cases, much much worse.

I don't think your links to philosophy and moral dilemmas provide as many answers as they create more questions.

Our societies morality is already codified in our laws - shocking, I know, but in many ways true. Turn that lot into java, stick it in a humaniform robot, and see what happens!

 

[srw]

Can a developer put in a "10cm rule" and do they? Yes, absolutely. Agile and DevOps can make this better or in most cases, much much worse.

They can, and they might. But as soon as driverless cars start wibbling about all over the place as a cyclist wobbles a bit, or (more likely) a loose dog is run over because the developer has forgotten about pets, the algorithm will rapidly be changed.

 

[PK99]

Can a developer put in a "10cm rule" and do they? Yes, absolutely. Agile and DevOps can make this better or in most cases, much much worse.

I don't think your links to philosophy and moral dilemmas provide as many answers as they create more questions.

!

/there is a whole branch of moral philosophy dealing with such questions: Trolleyology

The initial Trolley problem:

The trolley problem is a thought experiment in ethics. The general form of the problem is this:

There is a runaway trolley barreling down the railway tracks. Ahead, on the tracks, there are five people tied up and unable to move. The trolley is headed straight for them. You are standing some distance off in the train yard, next to a lever. If you pull this lever, the trolley will switch to a different set of tracks. However, you notice that there is one person on the side track. You have two options:

1. Do nothing, and the trolley kills the five people on the main track.
2. Pull the lever, diverting the trolley onto the side track where it will kill one person.

Which is the most ethical choice?

and
http://www.bbc.co.uk/programmes/b04grcnd

 

[jarlrmai]

Yes it's interesting isn't it imagine a world where all vehicles were automated and had to follow the current law (because the programme told them to.) It would be a different driving environment for sure. One where certain business models might not be so viable.

In the US for instance it is 'illegal' for vehicles to park in certain places for instance say in dedicated bicycle lanes this is nominally to make them safe for bicyclists to use, however as punishment for this is just a fine the delivery companies (FedEx, UPS et al) accept it, their drivers park in the bike lane and the company adds the fine to the cost of doing business, in fact they have a bulk fine paying systems setup, the driver hands in the tickets, the company pays them and the local authority has the income stream. Not every infraction is noticed and fined as there are limited enforcement officers so this "bribe" is kept a manageable level and the companies are able to deliver 5 parcels an hour rather than 1 if they had to find a legal parking place and delivery costs are kept low. I would imagine if you asked the companies if the official direction give to the drivers was to "park wherever you can illegal or not and we'll pay the fine." they would probably deny this I'm sure it's always just the driver doing it on their own

Now imagine the company wants to get rid of those expensive to pay and insure human drivers and get a fleet of self driving vans that park up nearby, text the deliveree who opens the parcel slot on the side of the van with a one time passcode from the text. Imagine also that in the future to ensure all self driving or otherwise vehicles know what to expect from other self driving vehicles your vehicle control code has to be open source or sent to a government body or otherwise become generally available. All of a sudden you have to programme this behaviour or by not programming it in it becomes an obvious and unforgivable oversight, you essentially have to write down in an official document your intent to break the law surely this isn't allowed, treating each drivers infraction as a seperate instance for which you have paid your fine is one thing but actually programming in the intention is different.

So now your problem is that bike lanes and cyclists exist and you have declare officially your intention to ignore their safety to keep your business model viable. So maybe you start to lobby the politicians, set up a few focus groups and engage with some journalists to see what you can do about this problem.

 

[theclaud]

/there is a whole branch of moral philosophy dealing with such questions: Trolleyology

The initial Trolley problem:

The trolley problem is a thought experiment in ethics. The general form of the problem is this:

There is a runaway trolley barreling down the railway tracks. Ahead, on the tracks, there are five people tied up and unable to move. The trolley is headed straight for them. You are standing some distance off in the train yard, next to a lever. If you pull this lever, the trolley will switch to a different set of tracks. However, you notice that there is one person on the side track. You have two options:

1. Do nothing, and the trolley kills the five people on the main track.
2. Pull the lever, diverting the trolley onto the side track where it will kill one person.

Which is the most ethical choice?

and
http://www.bbc.co.uk/programmes/b04grcnd

TMN to @srw...

 

[theclaud]

Well - that one went well, didn't it?

That's all jolly good, but I'm not sure that it demonstrates that there's no longer a problem with lorries killing people and various interested parties and commentators attempting to shift responsibility onto the victims.

 

[srw]

That's all jolly good, but I'm not sure that it demonstrates that there's no longer a problem with lorries killing people and various interested parties and commentators attempting to shift responsibility onto the victims.

It doesn't, but it wasn't supposed to. It was supposed to demonstrate that there isn't an inevitable link from lorries killing people to lorry-operators successfully bullying cyclists out of the way.

 

[jefmcg]

This week's Radio Lab this week is about self driving cars and the trolley problem.

 

[theclaud]

It doesn't, but it wasn't supposed to. It was supposed to demonstrate that there isn't an inevitable link from lorries killing people to lorry-operators successfully bullying cyclists out of the way.

Well that's alright then. Ensuring that the bullying-out-of-the-way-and-victim-blaming strategy fails will depend on actively resisting it, not on complacently assuming it just won't work and everything will be OK.

 

[theclaud]

Why are we discussing self driving software on page 97 (!?!) of a thread about a collision that didn't even involve a car.

This week's Radio Lab this week is about self driving cars and the trolley problem.

If you can't beat 'em...

 

[Wobblers]

They can, and they might. But as soon as driverless cars start wibbling about all over the place as a cyclist wobbles a bit, or (more likely) a loose dog is run over because the developer has forgotten about pets, the algorithm will rapidly be changed.

No. No it won't.

There is an enormous amount of work required to certify any safety critical bit of code. Not least that it has to be exhaustively tested, again, to make sure nothing undesired has been added. A large number of regulatory hurdles have to be cleared - and a very specific process followed. A rather large volume of documentation has to be produced to show that the correct process was followed.

This is not "rapid". Nor is it cheap. It is very much in the manufacturers' interest to try and deflect blame onto the cyclist. Indeed, a careful examination of a sufficient number of collisions should show that there is a fault. But... that is of little comfort to the victims - or, more likely, their friends and families. Worse, it requires an organisation with sufficient knowledge and skills in software and systems engineering to be able ask the appropriate questions, and come to the correct conclusions. At the moment, RTCs are investigated by individual police forces. None have the required technical skills - take a look a the accident reports produced by the AAIB or NTSB into air crashes to get a feel of the depth of the investigation required for these sorts of incidents. Further, unlike the air industry, there is no mecfhanism for reporting near misses or accidents. Without this, many - most in all probability - will be missed. And even when it is, belatedly, understood that there's a problem, it will require political will to release sufficient resources to conduct a proper well funded and in depth investigation that is required. (Don't forget, a careful inspection of the code will be required - proprietory and commercially sensitive information that the manufaxcturers are unlilkely to hand over without a court order or similar legal sanction.)

Which leads to the real problem: it is far easier just to blame the cyclists. After all, the autonomous car must be right - that's how it was programmed, wasn't it? Besides, cyclists are scoflaws wobbling through red lights - they're obviously a danger, it must be their fault. Don't believe me? Well then, just look at all the comments blaming cyclists underneath any news report of an accident involving one of them. It is far easier, and will even generate political capital from the, ahem, entitled motorist faction, to legislate against cyclists. They'll have their excuse: it's for their own protection. And this is exactly what I fear will happen.