Neural Nets to Be Used for Vehicle Control

Elon Musk has provided further guidance on Tesla’s next major release of FSD Beta, v11.3. Last week Musk said that v11.3 would be ready in about ‘two weeks’ and would contain many major improvements.

We’ve heard these two-week estimates before, but it’s reassuring that Musk is being specific with details on this upcoming release.

Now just seven days after his last tweet, Elon Musk is giving us more details on this significant upgrade.


musk said he expects FSD Beta v11.3 to start rolling out to some customers later this week, or next week at the latest. This matches up with his initial two weeks estimate a week ago.

Elon Musk’s estimates are known to be overly optimistic, but given the number of details he’s releasing, it sounds like Tesla may be close to releasing this next build.

FSD Beta v11.3 will likely roll out to Tesla employees first and then go out to select FSD Beta testers, possibly the original OG group. Since this is a major milestone and includes major improvements, expect a slow and gradual rollout. Although we may get our first glimpse into FSD Beta v11 this week, it may be several weeks or more before the majority of customers have access to this beta.

Neural Nets for Vehicle Behavior

A week ago Musk said this upgrade will include ‘many major improvements.’ Last night Musk revealed some additional details. He said there will be “many small things,” one of which will be that Tesla will begin to use neural nets for vehicle navigation and control, instead of just vision.

Today Tesla uses neural networks to determine the vehicle’s surroundings, where objects are, what they are, and their distances from the vehicle to create a 3D environment known as ‘vector space.’ With this information, the vehicle can then plan a path and navigate around these objects toward its destination.

However, based on Musk’s comment, it sounds like Tesla is currently only using neural nets to determine its environment and not for controlling the vehicle. This means that how the vehicle behaves, how it finds a path, and how it moves is still a process that is coded traditionally.

In the same way that Tesla uses millions of images to determine what a stop sign or traffic cone is, it sounds like Tesla will now use a large number of examples to determine how to best control the vehicle in various situations.

That could mean that Tesla will take millions of quality examples of how to gradually accelerate or slow down, based on real driving behavior to determine how the vehicle should accelerate in different situations.

This could be applied to every driving characteristic such as turning, slow downing, driving around a parked vehicle, etc.

If Tesla starts leveraging neural nets to aid vehicle control we may soon see drastic improvements to vehicle behavior, making it much smoother and human-like.

New Features

Although Elon Musk didn’t specifically mention new features coming to FSD Beta v11.3, there are several that have been talked about in the past that could show up in this major full self-driving update.

Reverse Creep

Reverse Creep has been a feature that has been talked about as far back as FSD Beta 10.13. This feature would allow the vehicle to go into reverse to move out of the way of danger or adjust its trajectory. Right now FSD Beta will only ever move forward, so this improvement would be a giant step toward achieving human-like behavior.

A good use case for this is when the vehicle moves forward for better visibility. There may be times when the vehicle sees a car coming after moving creeping forward. In these cases, it’d be smart to let the vehicle reverse back to its previous position if it is now in the path of traffic.

Navigating Without Map Data & GPS

In the past Musk also alluded to the fact that Tesla is working on the neural networks’ ability to complete ‘dead reckoning’ navigation, which is navigating based only on inertial measurements such as speed, direction and wheel movement.

He gave underground parking garages as an example of where FSD would need the ability to navigate without GPS or map data.

The car will be able to do this by using its last known GPS location and then determine its future location using only a compass, wheel movement and speed.

FSD Beta v11

FSD Beta v11 has always been expected to be a big leap forward, and as we get closer, that hasn’t changed. This update is expected to be a huge improvement to what is currently available to customers. Although Musk’s timelines have usually shifted and features have typically taken longer than initially planned, it looks like we may be getting close to the next major release for FSD Beta.

Although we’re still years away from true full self-driving, Tesla’s mission inches closer with every update.

Recently there was also a leak revealing some details of Tesla’s upcoming cameras in hardware 4.0, which are expected to include a fan and heater for select cameras.

Sonic The Hedgehog is returning to Teslas

One of our forum users, Thib, confirmed with Tesla Service that the beloved Sonic The Hedgehog game will be returning to Tesla vehicles.

According to Tesla Service, the game was only under a “temporary suspension.” The service representative goes on to state: “Indeed, this game is currently suspended, it will be re-integrated soon.”

Tesla added the Sonic game to its vehicles during the 2021 holiday update. It was only available for a year before Tesla removed it in the 2022 holiday update. The game required a USB controller to play.

Early last year, Tesla added Sonic to vehicles that did not have data-capable USB ports in the front via the glovebox. Version 2022.28.1 release notes read, “A game controller plugged into the glovebox USB port is required.”

Sonic the Hedgehog is a platform video game that was released in 1991 by Sega for their Genesis gaming system. Players control a blue hedgehog, named Sonic, to run at fast speeds on a mission to overtake Dr. Robotnik, a mad scientist who has locked away animals inside robot bodies while chasing after the Chaos Emeralds.

The removal of Sonic the Hedgehog is still unclear, but we’re speculating that it had to do with a bug or the license expiring.

Tesla also added Steam to a few of its vehicles, which signals that they may be transitioning away from adding more games. Instead, they’ll rely on Steam’s vast catalog of games for users to enjoy. Tesla may simply transition into making sure Steam is running smoothly on its operating system.

New information is leaked about Tesla's upcoming hardware 4.0 suite

We now have new information regarding Tesla’s updated cameras that will be used in Hardware 4.0. Twitter user @ChrisZheng001, who has ties with the Tesla supply chain in China, posted three images that appear to be camera configurations. While the information in the picture is limited, there is enough text to uncover what may be in development at Tesla.

Three Cameras to Two

The front-facing cameras will be reduced to two

Perhaps the biggest change comes to the trio of cameras that make up the forward-facing view for the Autopilot system. The translation appears to state that Tesla will be reducing its three front-facing cameras down to two and all cameras, including the interior camera, will be upgraded to ‘AP4’ cameras. The newer cameras are expected to have a much higher resolution in addition to other improvements such as LED flicker mitigation.

The only information provided by @ChrisZheng001 in the post is: The Autopilot HW 4.0 front camera will be reduced to two Cz (because) of the significant increase in both pixel density and field of view.

In November of 2022, it was revealed that a Tesla supplier was producing 5-megapixel cameras. The current cameras are 1.2 megapixels. So, while the number of cameras is being reduced, Tesla Vision will actually improve.

Fans for Forward Facing Cameras

The front-facing cameras will feature a fan to help with condensation

That leads to the next change. The housing size doesn’t appear to change, just what is in it. According to the document, and again using Google translate, it states: the shape of the black border at the position of the front camera has changed/added a camera fan.

By that wording, it sounds like the new housing will include a fan, likely to prevent condensation buildup. However, it seems open for change with a window for feedback and further evaluation.

B-Pillar Changes

B-pillar cameras will gain a heater in addition to a new camera opening

There is one more picture. The text reads: B-pillar exterior trim. B-pillar exterior panel camera change, opening shape change, new camera heating device. Inspection station: final care exterior decoration inspection appearance; heating function will be evaluated after project feedback.

There is not much to figure out there. Tesla appears to be adding a heating device to keep the B-pillar cameras clear and changing the shape of the opening. That could be due to the size of the new cameras or the field of view they require.

It’s widely speculated that a new high-resolution radar unit is in the works. That radar combined with these high-powered cameras means Tesla’s already incredible Autopilot system will only improve.


Leave a Comment

%d bloggers like this: