User Tools

Site Tools


synthetic_data_files

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
synthetic_data_files [2020/08/02 18:27]
bob created
synthetic_data_files [2020/08/14 11:53] (current)
bob
Line 7: Line 7:
   * Real flight test data is now just about impossible to get. Almost all flight test data is restricted in one fashion or another. So someone who wants to develop applications or just learn about flight test data is severely restricted in the data they can look at. Some sample data files are [[sample_data_files|available for download]] but these are laboratory recordings and are not realist data like you might see in a flight test.   * Real flight test data is now just about impossible to get. Almost all flight test data is restricted in one fashion or another. So someone who wants to develop applications or just learn about flight test data is severely restricted in the data they can look at. Some sample data files are [[sample_data_files|available for download]] but these are laboratory recordings and are not realist data like you might see in a flight test.
  
-  * Real flight test data doesn't always present //interesting// test cases. Big Data processing techniques are now being developed to analyze large sets of data. There is a lot of recorded flight test data but finding the right set of data to exercise some aspect of a Big Data search engine can be non-trivial.+  * Real flight test data doesn't always present //interesting// test cases. Big Data processing techniques are now being developed to analyze large sets of flight test data. There is a lot of recorded flight test data but finding the right set of data to exercise some aspect of a Big Data search engine can be non-trivial.
  
 Synthetic data solves both of these problems.  Synthetic data solves both of these problems. 
Line 16: Line 16:
 ===== Architecture and Approach ===== ===== Architecture and Approach =====
  
-Data is synthesized in several steps. Both GOTS and custom software applications are used to generate data files.+Data is synthesized in several steps. COTS, GOTSand custom software applications are used to generate data files.
  
-The GOTS [[https://www.dsiac.org/resources/models/bluemax6/|BlueMax]] software application is used to generate realistic flight path data. BlueMax allows designing a flight profile at a very high level of abstraction. From this flight scenario BlueMax calculates about 150 flight parameters such as aircraft position, attitude, speed, accelerations, angle of attack, throttle setting, etc.+  * The GOTS [[https://www.dsiac.org/resources/models/bluemax6/|BlueMax]] software application is used to generate realistic flight dynamics data. BlueMax permits designing a flight profile at a very high level of abstraction. From this flight scenario BlueMax calculates about 150 flight dynamics parameters such as aircraft position, attitude, speed, accelerations, angle of attack, throttle setting, etc.
  
-The output of BlueMax is read in to an SQL database to facilitate further processing.+  * The output of BlueMax is read into an SQL database to facilitate further processing.
  
-The BlueMax aircraft state data is then played back and used to drive the XPlane aircraft simulation software application. At each point in time a screen capture is made of the XPlane screen. The captured screen bitmap is the fed into a video encoder. The video encoder takes the series of screen bitmaps and converts them into a series of MPEG Transport Stream packets. These TS video packets are then written to the same SQL database as the BlueMax aircraft state data. The process can be repeated with different XPlane views to make multiple video streams. Video encoding is a very CPU intensive process. Precomputing the video data that goes along with the aircraft data makes the generation of IRIG 106 data files much more efficient.+  * The BlueMax SQL aircraft state data is then played back and used to drive the [[https://www.x-plane.com/|XPlane]] aircraft simulation software application. At each point in time a screen capture is made of the XPlane screen. Successive captured screen bitmap are then fed into a video encoder. The video encoder takes the series of screen bitmaps and converts them into a series of MPEG Transport Stream (TS) packets. These TS video packets are then written to the same SQL database as the BlueMax aircraft state data. The process is repeated with different XPlane views to make multiple video channels. Video encoding is a very CPU intensive process. Pre-computing the video data that goes along with the aircraft state data makes the later generation of IRIG 106 data files much quicker. 
 + 
 +  * A custom software application is then run to generate the synthetic IRIG 106 Ch 10/11 data files from the pre-computed aircraft state and video data. Starting date and time as well as a number of other input parameters are entered at run time to tailor the simulation. This application is the main simulation engine for generating the synthetic flight test data from the pre-computed aircraft state and from user inputs. This application is command line driven so that it can run in batch mode to easily generate large numbers of files. 
 + 
 +===== Current Capability ===== 
 + 
 +The synthetic data software suite is currently early in development. There is currently support for valid TMATS, a time channel, multiple video channels, and a 1553 channel with aircraft navigation data. Below is a screen image of the data being played back in [[https://telspandata.com/tm-ground-systems/data-fusion-display/|Telspan NetView]]. 
 + 
 +{{:synthdata_netview.png?direct&400|}} 
 + 
 +There is also a [[https://irig106.org/synth_data/2020-08-02_16-36-49.mov|screen capture movie]] showing about 1 minute of playback. The synthetic IRIG 106 Ch 10/11 [[https://www.irig106.org/synth_data/A-10_China_Lake_ILS.ch10|data file]] (about 150 MB) can also be downloaded. There is an [[https://www.irig106.org/synth_data/Synthetic%20Data%20Generator%20ICD.pdf|Interface Control Document]] describing the data channels and data fields in the data file. 
 + 
 +===== Future Plans ===== 
 + 
 +The near term development focus is to flesh out some other useful data content. 
 + 
 +  * PCM is an important data type. A PCM data stream will be developed to generate simulated strain gauge values. Strain gauge values will be generated based on aircraft dynamics such as G's and airspeed. A little bit of band limited noise will be added to make these simulated readings a little more realistic. 
 + 
 +  * Auto-generated 1553 will also be implemented. Realistic 1553 data channels have lots of RTs and subaddresses at various message rates. The auto-generated 1553 will make it easy to generate 1553 messages at various rates. 1553 message content will either be fixed or random. 
 + 
 +Longer term I will add the ability to easily modify the data for a simulated strain gauge PCM message or 1553 message in somewhat subtle ways. For example it would be an interesting search problem if one PCM channel became somewhat more noisy on one flight, or if the channel gain decreased significantly for one flight. 
 + 
 +~~DISCUSSION~~
  
  
  
synthetic_data_files.1596410861.txt.gz · Last modified: 2020/08/02 18:27 by bob