Matlab - 01

Working with Videos in MATLAB

MATLAB supports only "raw" (uncompressed) avi files on Linux and only some Indeo and Cinepack compressed versions on Windows. But mostly we get videos in MPEG. So we need to convert to raw or any supported format. For this, FFmpeg can be used.

AVI means Audio Video Interleave and is only a container format. AVI does not mean raw video. It is possible to have mpeg or many other compressed avi files. MATLAB cannot read such a compressed avi files. (See http://en.wikipedia.org/wiki/Audio_Video_Interleave).

1. Section: If we have raw video (or supported format)

If we have raw video, then handling in MATLAB is quite easy. We need to use functions mmreader, read, movie, mmfileinfo, frame2im, im2frame, aviread, avifile, aviinfo, addframe (avifile), close (avifile), movie2avi functions. Following code reads and plays back a given video. For further details and other functions see MATLAB help.

%Reads and plays back the movie file xylophone.mpg.

xyloObj = mmreader('xylophone.mpg');

nFrames = xyloObj.NumberOfFrames;

vidHeight = xyloObj.Height;

vidWidth = xyloObj.Width;

% Preallocate movie structure.

mov(1:nFrames) = struct('cdata', zeros(vidHeight, vidWidth, 3, 'uint8'),
'colormap', []);

% Read one frame at a time.

for k = 1 : nFrames

mov(k).cdata = read(xyloObj, k);

end

% Size a figure based on the video's width and height.

hf = figure;

set(hf, 'position', [150 150 vidWidth vidHeight])

% Play back the movie once at the video's frame rate.

movie(hf, mov, 1, xyloObj.FrameRate);

2. Section: If we have not supported video format

We can use any converter tool to convert compressed video to uncompressed avi format. One such tool is FFmpeg (http://ffmpeg.mplayerhq.hu/index.html). Mplayer, VLC and many other players are based on this codec.

If you are a Windows user, just download latest binary from http://ffdshow.faireal.net/mirror/ffmpeg/ (Any virus? I don't know. Download at your risk!). (You can use WinRAR to uncompress .7z - http://www.freedownloadscenter.com/Utilities/Compression_and_Zip_File_Utilities/WinRAR_Download.html )

Use this command to convert any compressed file (compressed.any) to uncompressed avi file (uncompressed.avi) (Note: pthreadGC2.dll file should be in the same directory as ffmpeg.exe).

ffmpeg.exe -i compressed.any -vcodec rawvideo uncompressed.avi

This should work most of the cases. However, sometimes it may not work (because, avi is a container format by Microsoft! :)). The converted video may have only blank frames or MATLAB may not be able to read it properly. In such a case, you can try steps explained in Section 3.

3. Section: If Section 2 does not work

Create a folder by name "images". Use this command to convert each frame of the compressed video to bmp (or ppm, jpg) images using ffmpeg or virtualDub and put it into the folder "images".

ffmpeg.exe -i compressed.any images/image_%d.bmp

Now you can use these images for processing. If you want uncompressed avi file, then use following MATLAB code to combine all raw images into a single uncompressed avi file.

%Script file to combine images to an uncompressed avi file

%Directory that contains images

in_dir = 'D:\temp\ffmpeg.rev11870\images\';

fout = 'D:\out.avi'; %Output file name

num_images = 341; %Number of images

%Set a suitable frame rate fps

aviobj = avifile(fout, 'compression', 'none', 'fps', 25);

for i = 1:num_images;

temp = sprintf('%d', i);

name = [in_dir, 'image_', temp, '.bmp']; %For ppm, change

img = imread(name);

frm = im2frame(img);

aviobj = addframe(aviobj,frm);

i

end;

aviobj = close(aviobj);

4. Useful Functions

mmreader, read, movie, mmfileinfo, frame2im, im2frame, aviread, avifile, aviinfo, addframe (avifile), close (avifile), movie2avi

5. Supported File Formats

Platform		Supported File Formats
Windows		AVI (.avi), MPEG-1 (.mpg), Motion JPEG 2000 (.mj2), Windows Media Video (.wmv, .asf, .asx), and any format supported by Microsoft DirectShow.
Macintosh		AVI (.avi), MPEG-1 (.mpg), MPEG-4 (.mp4, .m4v), Motion JPEG 2000 (.mj2), Apple QuickTime Movie (.mov), and any format supported by QuickTime as listed on http://www.apple.com/quicktime/player/specs.html.
Linux		Motion JPEG 2000 (.mj2), Any format supported by your installed plug-ins for GStreamer 0.10 or above, as listed on http://gstreamer.freedesktop.org/documentation/plugins.html, including AVI (.avi) and Ogg Theora (.ogg).

.

Model Based Design – VIP Appln -04

Model-Based Design to Develop and Deploy a

Video Processing Application

04 Implementing and Verifying the Application on TI Hardware

Using Real-Time Workshop® and Real-Time Workshop Embedded Coder, we automatically generate code and implement our embedded video application on a TI C6400™ processor using the Embedded Target for TI C6000™ DSP. To verify that the implementation meets the original system specifications, we can use Link for Code Composer Studio™ to perform real-time "Processor-in-the-Loop" validation and visualization of the embedded application.

Before implementing our design on a TI C6416DSK evaluation board, we must convert the fixed-point, target-independent model to a target-specific model. For this task we use TCP/IP Send and Receive a TI real-time communications protocol that enables the transfer of data to and from the host.

Creating the target-specific model involves three steps:

1. 
   
   Replace the source block of the target-independent model with the "TCP/IP Receive" block and set its parameters.
2. 
   
   Replace the Video Viewer block of the target-independent model with the "TCP/IP Send" block and set its parameters.
3. 
   
   Set up Real-Time Workshop target-specific preferences by dragging a block specific to our target board from the C6000 Target Preferences from Target Support Package library into the model.

Figure 7: Target-Specific Model.

To automate the process of building the application and verifying accurate real-time behaviour on the hardware, we can create a script, using Link for Code Composer Studio to perform the following tasks:

1. 
   
   Invoke the Link for Code Composer Studio IDE to automatically generate the Link for Code Composer Studio project.
2. 
   
   Compile and link the generated code from the model.
3. 
   
   Load the code onto the target.
4. 
   
   Run the code: Send the video signal to the target-specific model from the same input file used in simulation and retrieve the processed video output from the DSP.
5. 
   
   Plot and visualize the results in a MATLAB figure window.

Here is the Figure 8, shows the script used to automate embedded software verification for TI DSPs from MATLAB. Link for Code Composer Studio provides several functions that can be invoked from MATLAB to parameterize and automate the test scripts for embedded software verification.

Figure 8: Script used to Automate Embedded Software Verification for TI DSPs from MATLAB

Figure 9 shows the results of the automatically generated code executing on the target DSP. Means, you can observe that the application running on the target hardware properly, and verify that the application meets the requirements of the original model.

Figure 9: Result of the Automatically Generated Code Executing on the Target DSP

After running the application on the target, we may find that algorithm does not meet the real-time hardware requirements. In Model-Based Design, simulation and code generation are based on the same model, and so we can quickly conduct multiple iterations to optimize the design. For example, we can use the profiling capabilities in Link for Code Composer Studio to identify the most computation-intensive segments of our algorithm. Based on this analysis, we can change the model parameters, use a more efficient algorithm, or even replace the general-purpose blocks used in the model with target-optimized blocks supplied with the Embedded Target for TI C6000. Such design iterations help us optimize our application for the best deployment on the hardware target.

TCP/IP Setting Description

This demo is configured to run in an Ethernet network where there is a DHCP server. A DHCP server allows automatic assignment of the IP parameters of the target board. During this process, the status of the Ethernet link as well as the acquired IP address is displayed on the standard output window of the CCS IDE. If you don't have a DHCP server in your network, you must manually configure the IP parameters on the target side model. In order to do this, open the C6000™ IP Config block in the target side model, uncheck Use DHCP to allocate an IP address option and manually enter Use the following IP address, Subnet mask, Gateway IP, Domain name server IP address and Domain name parameters. You must always set Use the following IP address and Subnet mask parameters to meaningful values. As a rule of thumb, leave the Subnet mask parameter at its default value of "255.255.255.0" and set Use the following IP address parameter so that the first three numbers in the IP address assigned to the target board matches to those found in your computer's IP address. For example, if your computer's IP address is "100.101.102.3" you may choose "100.101.102.4" to be assigned to the target board provided that there is no other Ethernet device using IP address "100.101.102.4". Make sure that there are no IP address collisions when assigning an IP address to the target board. If you are unsure about what to enter for any other IP parameter, leave it at its default value.

Model Based Design – VIP Appln -03

Model-Based Design to Develop and Deploy a

Video Processing Application

03 Generating Automatic Code using Real Time Workshop

During simulation, the flexibility and generality provided by fixed-point operators as they check for overflows and perform scaling and saturations can cause a fixed-point model to run slower than a floating-point model. To speed up the simulation, we can run the fixed-point model in Accelerator mode. The Simulink Accelerator can substantially improve performance for larger Simulink models by generating C code for the model, compiling the code, and generating a single executable for the model that is customized to a model's particular configuration. In Accelerator mode, the simulation for the fixed-point model runs at the speed of compiled C code.

Code is generated using Real-Time Workshop in the folder named: modelfilename_mode_rtw (ex. MBD_LineDetection_CodeGen_accel_rtw), like below-

Figure 6: Generated Code with RTW

Model Based Design – VIP Appln -02

Model-Based Design to Develop and Deploy a

Video Processing Application

02 Converting the Design from Floating Point to Fixed Point

To implement this system on a fixed-point processor, we need to convert the algorithm to use fixed-point data types. In a traditional design flow based on C programming, this conversion would require major code modification. Conversion of the Simulink model involves three basic steps:

1. 
   
   Change the source block output data types. During automatic data type propagation, Simulink displays messages indicating the need to change block parameters to ensure data type consistency in the model.
2. 
   
   Set the fixed-point attributes of the accumulators and product outputs using Simulink Fixed Point tools, such as Min-max and Overflow logging.
3. 
   
   Examine blocks whose parameters are sensitive to the pixel values to ensure that these parameters are consistent with the input signal data type. (The interpretation of pixel values depends on the data type. For example, the maximum intensity of a pixel is denoted by a value of 1 in floating point and by a value of 255 in an unsigned 8-bit integer representation.)

This can be done using Fixed-Point Tool or by Fixed-Point Advisor from Tools->Fixed-Point

If this is first time, then start with Fixed-Point Advisor.

This opens a window as shown below:

The Fixed-Point Advisor is broken into four main tasks. Each task addresses one aspect of the conversion process. The tasks are:

Prepare Model for Conversion Evaluate model wide configuration options. Create floating-point base line data set. Prepare for Data Typing and Scaling Evaluate block specific configurations. Add design minimum and maximum information to the model. Perform Data Typing and Scaling Propose fixed-point data typing and initial scaling to the blocks. Analyze the resulting fixed-point model behaviour. Prepare for Code Generation Examine issues resulting in inefficient code.

After pressing the button "Run to Failure", tool will run till failure and then opens model advisor window as well as report like as below:

After doing the modification and changes need to run again.

Sometimes, to go directly to the error location we need to use "Explore Result…" Button as shown below.

Which open a window named Model Advisor Result Explorer.

Here select main signals to Log Signal Data from check box and Use Signal name for Logging name from pull up.

This Floating-Point to Fixed-Point conversion is an iterative procedure; we need to run every time after doing small changes in the model. And finally here we go with Fixed-Point Model.

Figure 4: Summary- No Fail.

Figure 5: Resulting Fixed-Point Model.

Model Based Design – VIP Appln -01

Model-Based Design to Develop and Deploy a

Video Processing Application

Purpose -

Model-Based Design with Simulink®, Video and Image Processing Blockset, Real-Time Workshop® and TIC6000 Blockset can be used to design a embedded system, to implement the design on a Texas Instruments DSP, and to verify its on-target performance in real time.

Why this approach?

The core element of Model-Based Design is an accurate system model— an executable specification that includes all software and hardware implementation requirements, including fixed-point and timing behavior. We use the model to automatically generate code and test benches for final system verification and deployment. This approach makes it easy to express a design concept, simulate the model to verify the algorithms, automatically generate the code to deploy it on a hardware target, and verify exactly the same operation on silicon.

Steps -

1. Building the System Model using Mathworks Simulink
2. Converting the Design from Floating Point to Fixed Point
3. Automatic code generation using RTW/RTW Embedded Coder
4. Implementing and Verifying the Application on TI Hardware

01 Building the System Model – Algorithm

Using Simulink, the Signal Processing Blockset (SP Blockset), and the Video and Image Processing Blockset (VIP Blockset), first develop a floating-point model of canny edge detection system. This model detects the edges of input video using a complex 'canny' edge detection algorithm.

Input a video stream to the simulation environment using the "From Multimedia File block" from the VIP Blockset. During simulation, the video data is processed in the Edge Detection subsystem, which outputs the detection algorithm results to the "To Video Display block" for video visualization.

The main subsystem of our Simulink model is canny edge detection. The sequence of steps in the edge detection algorithm maps naturally to the sequence of subsystems in the model.

We begin with a preprocessing step in which we define a relevant field of view and filter the output of this operation to reduce image noise. We then determine the edges of the image using the Edge Detection block in the Video and Image Processing Blockset. With this block we can use the Sobel, Prewitt, Roberts, or Canny methods to output a binary image, a matrix of Boolean values corresponding to edges.

Figure 1: Top Model in Floating-Point

Figure 2: Subsystem – Canny Edge Detection with Preprocessing

        

Figure 3: Results of Floating-Point

Model Based Design – Things to Know

Model-Based Design – Things to Know

Matlab Simulink and Stateflow based model design

The availability of sophisticated tools and new development techniques has provided great support to control engineers in overcoming these challenges. For instance, the model-based control design software, such as Matlab's Simulink and Stateflow, helps control engineers to recognize problems early on and reduce risk. With built-in mathematical functions and routines these tools are optimized for designing and analyzing control strategies through off-line simulation. Moreover these tools can be integrated with real-time hardware which means integrating traditional off-line simulation with real-world testing.

Benefits:

• Elimination of errors early in the design
• Robust control system
• Fewer iterations in the development cycle
• Reduced development time and cost

Design validation using Simulink simulation

Simulink facilitates designing of the control algorithm and also helps in executing off-line simulation on the desktop. But it doesn't mean that with software simulations all the distinctive behaviors of an actual dynamic environment can be accounted for.

Code Generation using RealTime Workshop or TargetLink

Tools such as Matlab's Real-Time Workshop (RTW) or dSPACE's Targetlink helps to create the prototype as well as the production code directly from the control design models. The greatest advantage of this process is the significant time and cost savings thanks to the inherent reproducibility and testability of the generated code.

Model-In-Loop validation (MIL)

The Model-in-the-loop simulation captures the specified behavior of the model that is to be implemented in C code later on. These simulated results are validated with the requirements. Also it acts as the verification reference for the next stages of development cycle. Since the model acts as the design document, it reduces the defects slippage due to translation of requirements to design.

Software-In-Loop validation (SIL)

The Software-in-the-loop simulation captures the behavior of the generated C code in simulator environment of the target controller. The simulated results when validated with the simulated Model-in-the-Loop simulation results both the results should be identical. If found otherwise the results can be used to evaluate the cause of deviation.

Automated Testing using Reactis, Embedded Tester, WinAMS, HEW Testing Tools

In the case of Simulink/Stateflow models the Software-in-the-loop simulation is improved by creating test cases using Reactis/Embedded Tester. These test cases not only simulate Simulink/Stateflow but also generate C code. This C code is then tested in WINAMS or HEW environment.

Hardware-In-Loop validation (HIL)

In hardware-in-the-loop (HIL) testing, the designer can verify the production system controller by simulating the real-time behavior and characteristics of the final system without the physical hardware or operational environment. While the system is simulated in real-time on a test computer the control code can be run on the target controller hardware. Though it is possible to connect the target hardware with the actual engine, testing against a simulated engine offers several advantages. When compared to a physical plant, a desktop simulator, often called a hardware-in-the-loop (HIL) tester, is far more cost-efficient, and easier to reproduce. The simulated engine also can simulate a variety of operating conditions or even fault conditions, such as engine stall, that would be difficult, costly, and/or dangerous with the actual plant. If measured data from HIL simulation deviates from Model-in-the-Loop simulation, the most likely cause is a bug in the target compiler or a problem with the processor.

Reverse Engineering of legacy code to Simulink models

The existing legacy C code is converted to equivalent Simulink/Stateflow models. This acts as specification for the legacy code, and maintenance of these specification will be easier, which may lead to auto code generation possible for future updates.

 

References:

1. http://blogs.mathworks.com/seth/2009/05/13/what-does-model-based-design-mean-to-you/
2. http://www.mathworks.com/model-based-design/#
3. http://www.mathworks.com/help/toolbox/simulink/gs/brc32o8-2.html
4. http://www.embedded360.com/model_based_development/model_based_development.htm
5. http://electronicdesign.com/article/eda/model-based-design12086.aspx
   

In the next blog, we will see in detail with an example of video analytics

Model Based Design (MBD)

Model Based Design of Embedded Systems

Target platforms needed to be the first choice when developing systems using assembler. Moving on to C, designers had more flexibility. But they were quickly locked into a specific target as the run-time libraries prevented migration to different platforms. Platform choice was less of an issue when there were fewer targets available. Yet these days, the options are mindboggling. It's no longer just a choice between processor A and processor B. Digital signal processors (DSPs), digital signal controllers (DSCs), multiple core processors, and even FPGAs are viable targets for embedded systems. With embedded solutions, it is target. Optimizing system cost, performance, and flexibility means choosing the proper target. But these days, software is such an important part of the solution, the choice of target and software tools is tied together? means, unless you use abstraction to disconnect these choices. This progression leads to model-based design.

Model-Based Design (MBD) is a mathematical and visual method of addressing problems associated with designing complex control systems. Model-Based Design is a process that enables faster, more cost-effective development of dynamic systems, including control systems, signal processing, and communications systems. In Model-Based Design, a system model is at the center of the development process, from requirements development, through design, implementation, and testing. The model is an executable specification that is continually refined throughout the development process. After model development, simulation shows whether the model works correctly.

When software and hardware implementation requirements are included, such as fixed-point and timing behavior, you can automatically generate code for embedded deployment and create test benches for system verification, saving time and avoiding the introduction of manually coded errors. Automatic program synthesis allows the generation of C code or a representation in a hardware description language (HDL). The HDL emulation can then be co-simulated in the system context to study behavior of an implementation at a cycle accurate level. This reduces expensive hardware iteration, facilitates analysis of system characteristics with detailed component implementation models, and mitigates the need for extensive test bench design.

MBD provides an efficient approach for establishing a common framework for communication throughout the design process while supporting the development cycle ("V" diagram). In Model-based design, development is manifested in these four steps:

• Modeling a plant,
• Analyzing and synthesizing a controller for the plant,
• Simulating the plant and controller, and
• Integrating all these phases by deploying the controller.

The model-based design paradigm is significantly different from traditional design methodology. Rather than using complex structures and extensive software code, designers can use MBD to define models with advanced functional characteristics using continuous-time and discrete-time building blocks. These built models used with simulation tools can lead to rapid prototyping, software testing, and verification. Not only is the testing and verification process enhanced, but also, in some cases, hardware-in-the-loop simulation can be used with the new design paradigm to perform testing of dynamic effects on the system more quickly and much more efficiently than with traditional design methodology.

Figure 1: Development Cycle ("V" diagram)

 

The important steps in MBD approach are:

1. 
   
   System identification (modeling the plant) an iterative process: -     By acquiring and processing raw data from a real-world system and choosing a mathematical algorithm with which to identify a mathematical model, the plant model is identified. Various kinds of analysis and simulations can be performed using the identified model before it is used to design a model-based controller.
2. 
   
   Controller analysis and synthesis: -    The mathematical model conceived in step 1 is used to identify dynamic characteristics of the plant model. A controller can be then be synthesized based on these characteristics.
3. 
   
   Offline simulation: -    The time response of the dynamic system to complex, time-varying inputs is investigated. This is done by simulating a simple LTI or a non-linear model of the plant with the controller. Simulation allows specification, requirements, and modeling errors to be found immediately, rather than later in the design effort.
4. 
   
   Deployment: -    Ideally this is done via automatic code generation from the controller developed in step 3. It is unlikely that the controller will work on the actual system as well as it did in simulation, so an iterative debugging process is done by analyzing results on the actual target and updating the controller model. Model based design tools like VisSim allow all these iterative steps to be performed in a unified visual environment.

Figure 2: Design Approach

Model-Based Design solution include:

• Algorithmic design & modeling using Mathworks products
• "Translation" of Matlab/Fortran/C into Simulink Models
• Creating generic custom toolboxes
• Integration of legacy C-Code into Simulink blocks
• Automatic code generation using RTW/RTW Embedded Coder
• Verification & Validation

Model-Based Design allows us to improve efficiency by:

• Using a common design environment across project teams
• Linking designs directly to requirements
• Integrating testing with design to continuously identify and correct errors
• Refining algorithms through multidomain simulation
• Automatically generating embedded software code
• Developing and reusing test suites
• Automatically generating documentation
• Reusing designs to deploy systems across multiple processors and hardware targets

Challenges:

With regard to the designing of embedded systems there are several challenges that the control engineers face and they are:

• Delivering products to market faster at lower cost
• Delivering implementations for increasingly complex systems
• Producing superior quality products
• Fewer system prototypes
• Flexible enough to adjust to last minute changes