Stadtpilot: First fully autonomous test drives in urban traffic

The Stadtpilot project aims at autonomous driving on Braunschweig's inner city ring road. For this purpose, an autonomous vehicle called “Leonie” has been developed. In October 2010, after two years of research, “Leonie's” abilities were presented in a public demonstration. This vehicle is one of the first worldwide to show the ability of driving autonomously in real urban traffic scenarios. This paper describes the legal issues and the homologation process for driving autonomously in public traffic in Braunschweig, Germany. It also dwells on the Safety Concept, the system architecture and current research activities.


I. INTRODUCTION
In the past two decades the driving abilities of automated vehicles have progressed rapidly. Especially the Grand Challenges sponsored by DARPA pushed the capabilities of autonomous vehicles to a new level. In 2007 the Technische Universität Braunschweig took part in the DARPA Urban Challenge, mastered all qualification rounds and was placed 7th as the best non-American team in the finals [1].
The follow-up project Stadtpilot has now taken the next step by driving autonomously in a real urban environment with all the additional challenges urban traffic provides [2]. In 2010 the German Aerospace Center (DLR) joined the team. After about two years of development the Stadtpilot showed first autonomous driving maneuvers on public roads among regular traffic in October 2010. This public demonstration was performed on the northeastern part of the inner city ring road of Braunschweig where the autonomous vehicle "Leonie" successfully showed lane keeping, the adaption of distance and speed to the traffic flow, traffic light interaction and a U-turn on an intersection. The road on this part of the city ring has at least two lanes for each direction and multiple intersections the vehicle had to pass.
Although the project aims at driving along of the 11 km city ring road, driving on this 2.5 km part already includes many of the possible traffic situations representative for the entire ring road. One major difference to the Urban Challenge is the interaction with many diverse traffic participants. Not only cars, but trucks, motor cycles, bicycles, pedestrians, T and many more occur, and the drivers are nos stunt drivers, but common traffic users. As they behave unpredictably and do not always follow the traffic regulations, driving here is more challenging and dangerous than on the enclosed Urban Challenge test track.

A. Related Work
As early as the mid-nineties, some research teams like those at Universität der Bundeswehr München, Carnegie Mellon University and University of Parma showed first autonomous driving on highways [3]- [5]. All teams used vehicles with a vision-based perception, but no digital maps nor GPS. Further important milestones in the development of autonomous vehicles were the DARPA Grand Challenges. They took place in the Mojave Desert in 2004 and 2005 where the contestants had to follow a single path defined by GPS coordinates [6]. With the announcement of the Urban Challenge the DARPA increased the degree of complexity in autonomous driving even further. During the final event in 2007, the vehicles had to prove themselves in a simulated urban environment [7].
The research activities started by this third part of the Grand Challenge series were continued by very interesting projects. Junior 3 [8] was built up by America's Electronics Research Lab of the Volkswagen Group with sensors closeto-production to automate applications like valet parking. Fully autonomous vehicles are also being developed by Google [9]. During the VisLab Intercontinental Autonomous Challenge [10] autonomous vehicles drove several thousand kilometres through mostly unknown terrain.
Over the last years, communication has become a very important feature for autonomous and assisted driving. This was demonstrated in projects like PRE-DRIVE C2X [11], sim TD [12] and the Vehicle-Infrastructure Integration (VII) test bed in California, USA [13]. These activities were supported by the approved IEEE 802.11p standard.

B. Outline
Section II describes legal issues and related safety functions, including the Safety Concept, development and test procedures as well as the packaging of software releases. In Section III we describe the demonstration scenario and the system controlling the vehicle. The ongoing research is introduced in Section IV, before Section V closes with conclusions and presents an outlook on future work.

II. LEGAL AND SAFETY ISSUES
Basically, there is no legal framework which enables autonomous driving on public roads. In Germany, autonomous driving contravenes the Vienna Convention on Road Traffic [14], where in Chapter I, Article 1 a driver is defined as follows: "(v) "Driver" means any person who drives a motor vehicle or other vehicle (including a cycle), or who guides cattle, singly or in herds, or flocks, or draught, pack or saddle animals on a road;" Consequently, the driver has to be a (legal) person and not a robot. He will also be legally responsible for the vehicle guidance and the compliance with traffic regulations. Additionally, the following requirements have to be fulfilled as stated in Article 8: "1. Every moving vehicle or combination of vehicles shall have a driver." "5. Every driver shall at all times be able to control his vehicle or to guide his animals." Therefore, "Leonie" must still have a specially trained safety driver who can always gain control of the car and monitors the traffic flow. This supervising is very important in urban traffic. There is a high hazard potential because of high relative velocities of traffic participants and narrow lanes. Furthermore, there are vulnerable road users such as motorcyclists or pedestrians without a crumple zone. Even a minor wrong movements or a jerk of the steering actuator could cause a very critical situation.

A. Safety and Control Concept
To receive an exceptional permission for autonomous driving without contravening the Vienna Convention, an official Safety Concept has been developed in cooperation with TÜV Nord. Among other things TÜV Nord is responsible for exceptional licensing of experimental vehicles from car manufacturers and suppliers in Germany. The Safety Concept allowing to perform autonomous driving on public roads has received the approval from the Lower Saxony state government. This concept contains requirements regarding documentation, safety and utilization of the test vehicle. In particular, dynamic limits on the actuating variables of the vehicle are defined. Regarding the vehicle's lateral movement the maximal steering wheel angle, the steering angular velocity, and the maximal steering wheel moment are limited. As a result, the effects of a possible malfunction are reduced.
In order to ensure the necessary dynamics of the vehicle in urban traffic, the steering wheel angle has been limited as a function of the vehicle speed. To find the specific limits, the steering wheel angle was recorded while driving manually in urban traffic. Fig. 1 shows the taken measurements in red and the empirically defined limits in blue. Subsequently, the limits were evaluated at different vehicle speeds at a test site, whether the driver was still able to control the vehicle during a step steer input or not.
Another important issue is the activation of the controlling system in the vehicle. Therefore, a key switch next to the gear shift serves as the main switch. The emergency button next to it can be used to disable the system as well. For testing and safety purposes the implementation allows activating lateral and longitudinal actuators separately as well as both combined. A standard push button on the steering wheel was rededicated to activate the system when being pressed for three seconds. This prevents from activating the system accidentally. The system can be activated either while the car is in parking position with activated electric parking brake or while driving manually on the desired course.
Listed below are additional preconditions that have to be fulfilled before the autonomous mode can be activated.
• The driver seat is occupied to assure a driver is present. • The driver is buckled up for legal and safety issues. • All doors including the hatch door are closed to avoid injuring entering and exiting passengers. • The ESC system is enabled to use the vehicle's assistance while driving. • There's no torque on the steering wheel to avoid injuries while turning the wheel automatically. • Accelerator and brake pedal are not pushed to assure the driver is not interfering. • The gear shift is in Position "D" so the car can drive on after leaving autonomous mode. • The voltage of the vehicle electrical system is above a certain level to assure voltage supply for all systems. • The estimated GPS position error is less than 40 cm to assure a high accuracy of the position. • The calculated lateral deviation from the planned course is less than 40 cm to avoid sudden correction maneuvers. • The black box recorder is properly logging the necessary data to document the vehicle's behavior. • All system components are transmitting keepalive signals with acceptable cycle times. If one of these preconditions is being violated during autonomous driving, the autonomous mode ends immediately and the safety driver recovers full control over the vehicle. This way, the safety driver has at all times the option to interfere in the vehicle's maneuvers and deactivates the system instantly by doing so.

B. System and Functional Tests
To assure a high level of software quality, a development process based on DIS/ISO 26262 part 6 [15] and the Scrum methodology is used [16]. This development includes the use of continuous integration and unit testing of all software components. After any change of code the whole system is rebuilt and all unit test cases are run.
Before driving on public roads, the chosen software revision is built and configured to test its behavior offline first. Afterwards the release candidate is deployed to all vehicle PCs and basic tests are run. They include activation and deactivation of the system and especially the safety driver's interference during autonomous maneuvers. All test results are documented in specially designed checklists as specified in the Safety Concept. The checklists also include testing the fixed integration of all additionally installed hardware as well as the self-diagnosis of the vehicle.
After completing this first testing stage, the release candidate is packaged individually for each involved PC. A detailed description of the packaging process can be found in section II-C. Next, the release candidate is evaluated on an enclosed non-public test site. For this purpose, different situations have been derived from the current milestone, e.g. approaching an intersection with different vehicle and traffic light configurations. If the release candidate accomplishes all those test cases, he finally gets cleared to control the vehicle in public traffic.

C. Software Releases
The Safety Concept demands that only tested software releases are used in public traffic. To comply with this rule within this very complex system, it is not sufficient to specify the software revision of the Stadtpilot code. Third-party software components or the operating system may change over time, too. Therefore, a software release management mechanism is essential as well as a strict separation of experimental and stable binaries and configuration files. This release management needs to tie all software versions together to ensure nothing can be changed undetected.
The Debian Linux operating system is used on every computer with our test vehicle. This suggests the usage of the Debian packaging system, which is widely used and accepted by the Linux community. Debian packages tie files and scripts together, sometimes with very complex dependencies on other packages. They can contain simply the need for another package, may only allow a specific version or forbid another package to be installed.
The project specific packages are built directly from the Stadtpilot source code revision system. On each PC one package is built and then checked for inconsistencies. The packages contain the necessary binaries as well as scripts for starting the services for autonomous operation. It also includes dependencies on the needed libraries in the version used to build the binaries. After packaging, the release candidate is tested according to the Safety Concept. If everything is working as expected, the package is archived and marked as stable release. By using the Debian packaging system, it is now possible to switch completely between the development and the stable status in a very reliable way in less than a minute.

III. FIRST MILESTONE
After two and a half years of development and testing and solving the legal issues, the team's efforts resulted in a first public demonstration. The achievements and the underlying hard-and software systems used in this demonstration are described in this section.

A. Public Demonstration
On October 8th, 2010 "Leonie" was presented to the public and drove autonomously on a selected part (see Fig. 2) of Braunschweig's northeastern inner ring road and mastered more than 40 km in numerous runs within four hours 1 . After handing over the control of the vehicle while driving manually on the desired course, "Leonie" conducted challenging driving maneuvers at speeds up to 60 km/h. The maneuvers included lane keeping, adjusting distance and speed to the flowing traffic autonomously, reacting to vehicles merging in "Leonie's" lane, and performing a U-turn without a single error or a dangerous situation. In addition, "Leonie" reacted to the current state of the next traffic signal, although at this stage of development the vehicle was not able to recognize the signal state automatically. For this reason the co-driver had to enter the current traffic signal state via an HMI. By the end of 2010, "Leonie" had completed several hundred kilometers during various test series.

B. Hardware Architecture
The experimental vehicle "Leonie" is a Volkswagen Passat Station Wagon modified by the Volkswagen AG. Fig. 3 shows the system components and their location in the vehicle. To enable control of all vehicle actuators via CAN, changes to ECU firmware were necessary (red) and additional hardware was installed like a brake booster enabling the electronic brake control (green).
The Vehicle Gateway is the main interface between the Stadtpilot Guidance and Control System and the vehicle. It communicates with the ECUs and the Stadtpilot system via CAN. Its main tasks are to receive the control messages, validate them and transmit them to the actuator ECUs and to thereby monitor the vehicle's state. In any case of failure or driver interference the Vehicle Gateway disconnects the communication between the systems and the vehicle control is completely transferred back to the safety driver.
Currently, the Stadtpilot Guidance and Control System is separated into three PCs with different tasks. The Control PC is an industry standard PC, mounted in a 19" rack in the trunk of the vehicle and equipped with an Intel Core 2 Quad processor and 4 GB main memory. It has four CAN interfaces for communication with the vehicle and sensors. As the localization is provided by an iMAR iTrace GPS/INS unit, the position accuracy is improved by Real Time Kinematic (RTK) correction data obtained via 3G network. For obstacle detection and tracking, a Hella IDIS2 LIDAR sensor with a 160 • field of view mounted below the front bumper is used. Additionally, the actuator control messages are generated and transmitted to the Vehicle Gateway to implement driving maneuvers.
The Visualization PC mounted in the glove box is connected to a touchscreen in front of the co-driver. It visualizes sensor data, the current vehicle state and receives user input as well. It is possible to change the desired driving states like parking or driving, to select the desired course and to input traffic light states if a traffic light on the course does not use C2X communication. This PC is also connected to the vehicle's audio system to inform the passengers about important events and errors.
To fulfill the safety requirements a Blackbox Recorder PC is used to record all data from CAN, the sensors and the network communication between the PCs. It is also recording two camera streams from the front and the rear of the car. All PCs are linked together via Gigabit Ethernet and are supplied by a secondary battery, which is charged by an additional generator.

C. Software Architecture
As mentioned in Section II, we use Debian Linux as the operation system within the vehicle since it is known for stability and can be customized easily. Necessary hard realtime capabilities are provided by preemptRT which supports cyclic method calls with very low latency as a Linux kernel enhancement and is easy to use. Another lesson learned during the DARPA Urban Challenge is the benefit of using established software frameworks e.g. for communication instead of creating and maintaining them.
The commercial Automotive Data and Time-triggered Framework (ADTF) serves as the basis of the Stadtpilot software. It provides an application framework based on the pipes and filters pattern. Therefore, every software component is implemented as an ADTF filter with a specified number of input and output channels. Within a graphical configuration editor, filters can be connected and can exchange data from now on. Thus, single ADTF filters or complete configurations can be reused in related projects [17]. Furthermore, ADTF offers connectivity to common communication interfaces, e.g. CAN, components for powerful 2D and 3D visualization, and record/replay functionalities.
The real-time communication framework RTI DDS is used for every communication between ADTF applications and computers. RTI DDS is interoperable by implementing the OMG DDS standard. The DDS communication is specified by an Interface Definition Language (IDL) file and a set of quality of service parameters. Since Stadtpilot relies on Intel CPUs, the Intel C++ Compiler Suite and the Intel Performance Libraries are also used. These packages offer great improvements of algorithms by using auto vectorization and very processor customized standard functions. Speech synthesis is made by a text-to-speech engine for automotive applications by Nuance. Thereby, the vehicle is able to communicate important information and warnings in a very natural and non-distractive way.
A central element of the overall system architecture from October 2010 shown in Fig. 4 is the Environment Information System (EIS). It combines all incoming LIDAR tracks with the ego position and a digital map. The Map consists of information about lane boundaries, optimized trajectories and speed limits for each lane as well as traffic light positions. This way, the EIS is able to derive the next traffic light from the Map and retrieves its state via the HMI. After matching the tracks with their corresponding lane, this information is forwarded to a simple AI module. The AI then chooses the lane and the object (track or traffic light) to follow. The corresponding trajectory is transmitted to Control module which adjusts the distances to the chosen object and keeps the vehicle on the trajectory. Also the monitoring functionalities described in Section II-A are placed here.

IV. ONGOING RESEARCH
To enable "Leonie" to drive on more parts of the ring road, the following work packages are being worked on since the public demonstration last year.

A. Position Accuracy and Integrity
The presented system architecture is currently highly dependent on the precision and the integrity of the given vehicle position. Using expensive GPS/INS hardware does not guarantee sufficient results (see Fig. 5). Taking the inaccuracy of the given a priori information into account, a continuous horizontal positioning error σ pos < 0.1 m is desirable. Additionally, occurring vehicle position jumps should be avoided, too. Beyond that, Receiver Autonomous Integrity Monitoring (RAIM) algorithms do not prevent the GPS receiver from calculating wrong RTK solutions. Systems like lane detection could end this dependency, but they do not perform sufficiently in urban environments, yet.
To improve both accuracy and integrity of the positioning information, a landmark-based positioning system has been designed. The developed feature extraction module uses LIDAR measurements to extract the highly-reflective lane marking signatures. It also handles segmentation, thus delivering relative positions of lane marking elements (see Fig. 6). In the next step, the extracted features will be matched with a digital map, in this way computing absolute positions of the features. The feature detection and position correction modules show very promising first results regarding precision and robustness, so that they are currently being integrated into the vehicle and evaluated [18].

B. Complex Intersection Scenarios
Along with a dependable Car-To-X (C2X) communication technology applications like safer passing trough intersections and cooperative driving will be possible resulting in less accidents and lower emissions. To develop and test such systems, a test bed called Application Platform Intelligent Mobility (AIM) [19] will be installed including C2X infrastructure for IEEE 802.11p compliant networking alongside the Stadtpilot route. Therefore, Braunschweig's inner ring road will be equipped with C2X infrastructure in order to have a continuous wireless connection to at least one Road Side Unit (RSU) at any position on the track. These RSUs will be able to access the current state and phase information of adjacent traffic lights and transmit the information to vehicles via C2X. By receiving traffic light states the vehicle's speed can be modified to approach and pass intersections when the traffic light is green. Therefore, the remaining time of each phase is transmitted to the vehicle.
Regarding turning with oncoming traffic the vehicle's environment perception has to be improved and more a priori information has to be provided. Based on this data the reasoning mechanisms have to solve complex scenarios where a complete perception is not possible, e.g. when vehicles cover other vehicles on parallel lanes. To cope with complex intersection situations and lane changes the environment perception system from the CarOLO project is adapted and improved. All necessary sensors have already been integrated into the vehicle in compliance with TÜV Nord.

C. Lane Changing
The next development step aims at performing autonomous lane changes. Therefore, at least the same reliability in the environment perception as on intersections is necessary due to high relative speeds, line-of-sight obstruction of traffic members by others and a high hazard potential. Stable tracking of dynamic objects and matching them to the corresponding lane is a key issue. As the ring road has more than two lanes in each direction in some parts, also vehicles changing from the outer lanes to the autonomous vehicle's target lane have to be recognized. Additionally, decision parameters of humans initiating lane changes on urban roads have to be identified and then transferred into the artificial behavior of the vehicle.

V. CONCLUSIONS AND FUTURE WORK
The main aspects in the development and structure of the Stadtpilot system, which drove autonomously in urban traffic as one of the first projects worldwide and the first in Germany, have been presented. Successful test drives proved the reliability of the Safety Concept which thereby enables the development of autonomous vehicles for urban environments.
Further improvements of every part of the system are currently being developed. They are necessary to realize more complex driving maneuvers and reduce the dependence on GPS and a priori information. Thus, the fulfillment of the next project objective -mastering the complete city ring -will be within reach. Abilities like collision detection, mitigation and avoidance need to be implemented to improve the system safety and reduce the risk of hazards to a minimal level.