On Wednesday I took part in autonomous car racing league. Let me share the story.
I'll start with a trailer from Amsterdam:
What is it all about?
Amazon's AWS is organising a series of Summits - one day event with various talks and presentations about Amazon Web Services and products it offers. This year one of the items is AWS DeepRacer League - a race event for cars offered by Amazon, trained to drive using reinforcement learning. I'll leave it at that, have a look at my previous posts to learn more about them.
I've been preparing for the races for over a week and was very curious to see how I would do. AWS Summit London on the 8th of May 2019 was my chance to find out.
The rules
There are pretty much three types of people taking part in the AWS DeepRacer League:
- coincidental ones came over unprepared to see what it all is about. They were able to choose one of the pre-trained models that, let's be frank, were guaranteed to not give a competitive result.
- curious ones took part in the workshop and spend an hour or two training their own model. They did some prep work and wanted to put it to use straight away. In many cases it was also their last chance to try it with a real car, but at the same time some of them continue working on virtual tracks after the event
- prepared ones. They knew what it was about, what was at stake, spent time and resources to prepare to this. They did not want to walk away empty-handed
The rules were as follows:
- take your shoes off - the track need to be handed delicately
- you get your models loaded on a stand-by car
- you have four minutes to get a good lap
- a lap is considered valid if your doesn't go off the track more than two times
- you are accompanied by a helper who grabs the car and puts it back where it left the track
- you get to control the throttle, the rest is autonomous
- the fastest lap goes on the leader board
The best ten get a DeepRacer, top three get a trophy and a DeepRacer jacket and a winner gets a full expense covered trip to re:invent 2019 in Las Vegas to compete in the great DeepRacer League 2019 Final.
My day
I woke up early and selected two models to try: one stable but slow, other fast, but flaky. Got out of the house just past 6:00 am and reached ExCel at 7:00. Shortly after I got in and located the DeepRacer League Arena. It was still being set up so I had a tour around the expo area which wasn't fully prepared either. I gathered a couple leaflets, had a chat here and there and when I reached the tracks again, they were ready.
In simulated environments the flaky model would complete two-three out of five laps and one would be at around 15.6 seconds.
I may have mentioned I tried to load some models on the car and to get it going in the office but without success. I knew that there may be issues if the track is not replicated ideally, I also only had models from re:invent workshops and didn't know how well they were really trained. Many improvements got applied since then. My goals were therefore not very high: complete a single run and do it in under 40 seconds.
I was the first to give it a go. Trying to work out how to do it all took a lot of my attention, so I don't remember much. The car would fall off the track every now and then and would be put back in. The steering was strange as I had to look at the car and use two touch screen buttons in the middle of a tablet at the same time. The thing with throttling is that it is a good idea to limit it on turns as the car can only analyze 15 frames a second and in a quick run the car could struggle to detect the turns properly, then on the straight line it makes sense to speed it up. By the end I managed to get a full lap at 13.690 seconds. Apparently this was the fastest start of the day so far.
Just to give you an idea what it looks like (this isn't the fastes lap, I think, just the one I got recorded):
That was so exciting! Everything came together at the right time and worked like a charm. The folks at Amazon shared the enthusiasm. Suddenly there were cameras, microphones, reporters. The guys at Amazon are pretty serious about promoting their stuff!
After two or three hours others started showing up with prepared models and I started getting lower and lower. I went to a DeepRacer workshop in hope to get some advice that's not written in the workshops script. By the end of the workshop I already new a couple guys interested in the subject. I also went down to fifth. The best result at that time was 10.5 seconds, way below what I thought I could do.
I tried to train my models to do better but two tries I took at the workshop weren't too good, both significantly slower than the one I used. I started another training: I cloned the model used in the morning with an identical reward function and had it run for 80 minutes - enough to give me 30 minutes to queue up and try again. In the meantime I took the initial model and went to have another go. I hoped that just having the experience from the first attempt could improve my performance. I didn't do worse, but not better either - at 13.7 seconds I haven't progressed. By that time I was down to tenth.
I went back into the queue and waited for the training to complete. The guys from top places were already under 10 seconds and were pushing further, trying to win the top spot. I didn't count on that - just improving by a second would move me up by four places. That seemed doable. Once the model was ready, I copied it to a flash drive and started the evaluation. The choice was simple - if it seems better, I'll take it. If not, I'll stay with the previous one.
Out of five laps, the model completed four, one of them at 15.3 seconds - that's the potential I like, both faster at peeks and more stable. The choice of model was clear then: Centre4-and-cut-corners-straight-on-straight-triple-clone-3-1-2. Have I mentioned I had problems with naming?
As I approach the track, MattC is having his last try. I was already on the track when he did 8.917 seconds. We had a winner. Also someone jumped into the first ten and I was eleventh, and one of the last to try with only the last person trying his model from the workshops.
The car started going but kept falling out of the track at first. It appeared to be having problems with the two left turns after the right one. At some point it took the first left too early and went back on the start line. Another go - same thing, but this time it was too fast for the helper and returned to the track after the second left turn. I think this is when I started the two fast laps. They were excellent, the car cut corners nicely, stayed on track all the time, I could hear the crowd - they also saw I had something good in here.
Two nice laps and then the wall, and the time ended. And the result: 9.17 seconds and I'm third!
Just to give you a bit of a context: The lap is 17.67 meters long which gives average speeds in the area of 2 meters per second, or 7 kilometers per hour. You keep turning towards the car and after a couple loops your head starts spinning slightly.
I couldn't believe it, what a come back that was! I got to the podium, I got the cup and the jacket. And more cameras - quite a new experience.
I was exhausted too.
AWS DeepRacer TV Episode
Spiruel
That's the guy who had the last go in the AWS Summit London. He made it into the twentieth with 28.836 seconds. Awesome, isn't it? He had only started a couple hours earlier. I remembered him mainly because of something that many of us engineers are doing to ourselves: belittling our achievements. He kept calling his model poor (may have used a different word). I wanted to mention him because what he did was awesome - dude made the effort to try a new thing and to challenge it at such an early stage. How many of us hide such things.
A little message to everyone: Never belittle your achievements, even if it is a hello world in a new language. You've done more than many and while it might not get you very far just yet, you do need to pat yourself on the back for that.
It's coming home
My return home was very pleasant despite having to carry a huge and heavy cup all the way. First I had a chat with some people from the Summit at the venue, then on the train. Then strangers started approaching me, asking about the trophy and congratulating. To complement this, I met my neighbours and received congratulations.
The only thing that could make this evening even better would be Tottenham going into the Champions League final. Who would've thought!
Lessons learned
Some takeaways from Wednesday:
- Work out a better naming scheme
- Keep trying - randomness is part of the learning process and things may change for better or worse
- Be careful about the costs - I've gone through many hours of training and this costs quite a bit. Eventually I will run out of credit and it would be beneficial to find a smarter way to do this.
- Appreciate your small achievements
- Make sure to make friends along the way
DeepRacer London Group
Lyndon has had a great idea to form a support group where we could share experiences and help each other to get better at training our cars. If you would like to join, click here (this is a Slack invite link).
Next plans
I'm preparing myself for the virtual races in AWS DeepRacer League. Apart from the summits physical races, Amazon is preparing six virtual races. The first one called London Loop is already available. I will continue to learn about machine learning, AWS products, racing and everything associated with this. Be sure to follow me on https://codelikeamother.com or @codelikeamother on Steem.