Overview
he Cloning-DCB project aimed to develop visomotor models for autonomous driving trained by imitating expert drivers, including software agents with privileged simulation information and real human drivers. Beyond functional performance, it focused on explainability by analyzing model attention to image content and comparing it with human driver attention using eye-tracking. The advanced model CIL++ was trained in the CARLA simulator on fixed routes with programmed events to avoid monotony; its attention was contrasted with human drivers, showing significant correlations. From human attention insights, an improved training method produced Guided CIL++, which outperformed CIL++ in driving metrics and attended more comprehensively to key image elements. Throughout these developments, a new human driving-related dataset has been generated for scientific sharing. We call it LAIA (Labelled Attention for Intelligent Automobiles dataset).
LAIA is a dataset of synthetic driving sequences generated using the CARLA simulator. The driving tasks are performed by 40 individuals on both dynamic and static driving platforms. Driving is not random but based on orchestrated scenarios. Each sequence includes RGB images accompanied by standard ground truth data (depth, optical flow, semantic and instance segmentation), ego-vehicle information, and, most notably, eye-tracking recordings. LAIA may be used for research and commercial purposes. It is released publicly under the Creative Commons Attribution-Commercial-ShareAlike 4.0 license. For detailed information, please check our terms of use. LAIA features more than 15 hours of curated driving scenarios and free driving performed by human drivers in a realistic cockpit with CARLA simulator. 5 weather variations for every driving scenario are also provided.
Ground-Truth
LAIA Dataset brings per-pixel ground-truth semantic segmentation, scene depth, instance panoptic segmentation, optical flow and real eye tracking. Check some of our examples:
Download LAIA
The LAIA dataset was created from data collected during driving sessions on two simulation platforms using CARLA as the simulation software. The data was captured with three cameras—center, left, right—as well as the side mirrors. It includes not only RGB images and the driver’s driving maneuvers but also the driver’s focus of attention, obtained through eye-tracking to monitor gaze direction. Additionally, the dataset provides various ground-truth annotations such as instance segmentation and depth maps, along with driving sessions conducted under different weather conditions.
LAIA Dataset has been classified under 6 concepts:
- Driver (Platform)
- Map
- Route
- Data Modality
- Camera
- Weather
These concepts has been used to generate a folder hierarchy in 6 levels to generate the file name of each package containg the desired data, as depicted in Figure 2.
Driver (Platform)
A total of 44 drivers participated in the data collection. Each driver was assigned to one of the two driving platforms, ensuring that there are no overlapping drivers between the two groups. The table below summarizes the driver classifications.
Map and Route
During data acquisition, each driver completed several driving sessions along various predefined routes within a CARLA town. Each route consists of a series of scenarios, as detailed in the project paper. In our context, "TownXX" refers to a specific CARLA town, and "RXX" denotes a particular predefined route within that town. The table below summarizes the CARLA towns used for data collection, along with the number of routes and their respective references for each town.
| Town | Routes |
|---|---|
| Town01 | R01 |
| Town04 | R01, R02, R03 |
| Town06 | R01, R02, R03 |
| Town10 | R01 |
Data Modality
The data modality refers to the type of data captured, primarily from CARLA sensors and linked to a specific camera. The table below summarizes the naming conventions for each modality and explains their meanings.
| Data | Acronym | Description |
|---|---|---|
| Color | RGB | Pixel information obtained per camera. This requires to indicate a weather too. More info here |
| Depth | DEPTH | Pixel distance obtained per camera using a CARLA Depth camera at same position and resolution thaen the associated RGB camera |
| Semantic Segmentation | SS | Pixel-level semantic segmentation labels in color, in PNG format. We follow the 19 training classes defined on Cityscapes More info here |
| Instance Segmentation | IS | Pixel to class and instance classification. More info about decoding here |
| Optical Flow | OF | Pixel velocity. More info here |
| Attention (Eye Tracker) | AET | Eyetracker gaze data aligned to screen position |
| CAN Bus | CB | CANBUS data at every timestamp, at a frequency of 25 Hertz. We provide the data in a json file with the next structure:
|
Camera
Both simulation platform setups include five cameras (with their positions and resolutions): Central, Left, Right, MLeft, and MRight. RGB, Depth, SS, IS, OF, and AET sensors utilize the positions, orientations, and resolutions of these cameras to spawn sensors with matching properties.
The table below summarizes the naming conventions for each camera and explains their meanings.
| Camera | Description |
|---|---|
| Central | Cámera that projects in the monitor in front of the driver and simulates the central part of the field of view of the driver |
| Left | Cámera that projects in the monitor left of the central monitor and simulates the left part of the field of view of the driver |
| Right | Cámera that projects in the monitor right of the central monitor and simulates the right part of the field of view of the driver |
| MLeft | Cámera that projects in the lower right square of the left monitor and simulates the left side mirror |
| MRight | Cámera that projects in the lower right square of the right monitorand simulates the right side mirror |
Weather
The dataset contains the same driving sessions under different weather conditions. These variations only affect the RGB images, which were generated from the original sessions. Therefore, the driver’s behavior remains consistent across all conditions.
These are the CARLA weather conditions used: ClearNoon (default), ClearSunset, CloudyNoon, HardRainNoon, MidRainSunset, WetNoon.
Data packages
The dataset has been grouped and compressed to provide a set of .tar.gz files. The naming convention of the files es depicted in next image:
Once a file is decompressed, the hierarchy of folders is created too. Some file names an their content explanation has been depicted in next table:
| File name | Description | Folder path |
|---|---|---|
| D017_Town01_R01_RGB_Central_ClearNoon.tar.gz | RGB data from Central camera during the driving session of driver D017 performed on route R01 on CARLA town Town01. The weather was ClearNoon | ./D017/Town01/R01/RGB/Central/ClearNoon |
| D029_Town04_R02_AET_Right.tar.gz | Attention data from Right camera during the driving session of driver D029 performed on route R02 on CARLA town Town04. | ./D029/Town04/R02/AET/Right |
| D019_Town06_R03_CB.tar.gz | CAN Bus data during the driving session of driver D019 performed on route R03 on CARLA town Town06. | ./D019/Town06/R03/CB |
Data downloading
The following section is a tool to select and download the desired data packages. The first five dropdown menus allow you to choose the basic concepts (such as platform, map and route, Data modality, etc.). The last dropdown is a fine-tuning selector to add or specify additional packages. Please note that some data modalities may require extra information—for example, selecting RGB data might require specifying weather conditions, and choosing a camera may need additional details.
Once you've made your selections and ensured they are complete (for instance, selecting RGB without specifying weather will prevent the download button from being active), the "Download Links!" button will become accessible.
When you click the "Download Links!" button, you'll see a list of links from each of the two available mirrors. You can download files individually by clicking each link, or copy all links using the "Copy to Clipboard" button and then use a download manager, like JDownloader, to download everything at once.
The initial selection consists of all driving sessions on both platforms of route R01 in Town01. The camera is Central, and the data modalities include RGB, AET and CB. Lastly, the weather conditions are ClearNoon and HardRainNoon.
Data size considerations
The entire dataset consist of 7,336 packages, totaling approximately 14.296 TB.
Typically, a driver completed three driving sessions on different routes within the Dynamic platform. Drivers who operated on the Static platform completed four sessions in average. Each driver is associated with roughly 230 packages, with an average size of about 600 GB.
The largest packages are those related to AET, with an average size of 40 GB per package.
The longest driving sessions took place in Town01 (R01), with an average duration of 11 minutes. The RGB packages from these sessions, specifically under ClearNoon weather conditions, have a size of approximately 11 GB.
Terms of Use
The LAIA Dataset is provided by the Computer Vision Center (UAB).
CloningDCB Project may be used for academic and industrial research, and it is subject to the Creative Commons Attribution-Commercial-ShareAlike 4.0. A summary of the CC-BY-SA 4.0 licensing terms can be found here. For comercial purpoises please, contact contact@cloningdcb.org
While we strive to generate precise data, all information is presented 'as is' without any express or implied warranties. We explicitly disclaim all representations and warranties regarding the validity, scope, accuracy, completeness, safety, or utility of the licensed content, including any implied warranties of merchantability, fitness for a particular purpose, or otherwise.
Privacy Policy
CloningDCB.org uses Google Analytics to collect data. We need this data to understand how you use our website so we can improve its design and functionality.
With your consent, Google Analytics will process and collect your personal data (cookies and IP address) to give us valuable information. Google Analytics will transfer your data to the United States and store it for 2 months. To learn more about Google's data transfer policies, click here.
Standard contractual clauses are legal clauses written by the European Commission. They are part of a contract between Google Ireland Ltd. and Google LLC, and Google LLC must follow them. Standard contractual clauses tell Google LLC what it can and cannot do with your data.
You have certain rights over your data: for example, you can require us to delete them or to provide you with a copy. We take responsibility for the processing of your data. We are available to answer any question and handle any request from you. Please use contact@cloningdcb.org if you we want to get in touch with us.
Team
| Principal Investigators: | Antonio M. López, Aura Hernández |
| Rest of the team: | Abad, Rubén; Borràs, Agnes; Cano, Pau; Contreras, Ainoa; Gil, Débora; Levy, Alexandre F.; Porres, Diego; Sánchez, Carles; Sánchez, Gemma; Serrat, Joan; Villalonga, Gabriel; Xiao, Yi; |
Citation
TBD
Contact
If you have any questions about the dataset, please feel free to contact us using this email: contact@cloningdcb.org
Acknowledgements
- Funded by:
Project TED2021-132802BI00 funded by MCIN/AEI/10.13039/501100011033 and the European Union NextGenerationEU/PRTR
- Personal acknowledgments:
Antonio M. López acknowledges the financial support to his general research activities given by ICREA under the ICREA Academia Program. CVC's authors acknowledge the support of the Generalitat de Catalunya CERCA Program and its ACCIO agency to CVC’s general activities.
- Developed by:
