In the last post, we discussed priority inversions and how developers can minimize and avoid them in their RTOS based applications. Priority inversions are not the only potential issue that developers can encounter. Another potential issue that developers may encounter that can wreak havoc on their system is known as the deadlock (also called the deadly embrace).
Deadlock occurs when two or more threads are blocked waiting for a resource that is held by the other thread. Let’s discuss a simple example on how deadlock occurs. Consider two separate threads, thread1 and thread2. Each thread requires mutex1 and mutex2 to perform their function. Thread1 starts executing its code, acquires mutex1 and continues executing its code. Thread2, being a higher priority, interrupts thread1 and acquires mutex2. Thread2 requires both mutex 1 and mutex 2 to complete its function and now attempts to acquire mutex1 which is already locked by thread1.
Being unable to continue executing its code, thread2 yields the CPU to thread1 to continue executing its code. However, now thread1 needs to acquire mutex2 to complete executing its code in order to release mutex1. So now thread1 yields the CPU to other threads to execute while it waits for mutex2 to become available. The problem is that neither thread1 or thread2 will ever finish executing their code because they are waiting on a resource that will never become available and is held by the other thread. The threads stop executing, creating a deadlock condition where neither task is able to continue executing their code.
Obviously, having a deadlock occur in an embedded system, especially a safety critical system, could have very bad repercussions. Avoiding deadlock is quite simple. First, developers should review their software architecture and determine if there are any conditions that could cause the threads to lock-up. In their design review, developers should attempt to minimize the number of threads that require two or more resources to execute. If there are no threads that are dependent on two or more resources or that don’t share resources amongst themselves, then there is no opportunity for a deadlock to even occur.
Second, when developers review their design, if they discover two threads that are dependent on the same two or more resources to execute, they should setup those threads so that they acquire the resources in the same order. In the example that was described above, if both thread1 and thread2 acquired mutex1 and mutex2 in the same order, the thread that couldn’t acquire mutex1 would have yielded the processor and allowed the other thread to acquire mutex2 and complete their critical section of code. The result - no deadlock.
A third option that developers can use to help minimize the impact a deadlock would have on their software is to avoid using TX_WAIT_FOREVER as the timeout on their mutexes and semaphores that are protecting resources used by other threads. If a developer allows a timeout to occur and then releases the resources that are currently held, they can become available for other threads. Using the timeout method will not prevent the deadlock but will bound how long it can exist in the system. Essentially the thread realizes it cannot complete its execution and it releases the resource it is holding in the hope that the other thread holding the required resource will complete and yield those resources. This option doesn’t solve the deadlock issue but minimizes its impact.
Priority inversions and deadlocks are probably the most well-known issues that developers encounter when working with a real-time operating system. These conditions can be prevented by proper architecture design and carefully reviewing the design and code implementation for these issues. A less commonly discussed issue is thread starvation. In the next post, we will discuss what thread starvation is and how to avoid it.
Until next time,
Live long and profit!
Professor_IoT
Hot Tip of the Week
The first 10 Module Guides are now available on Renesas.com. These application projects include an application note plus a SSP project created specifically to demonstrate the use of a single SSP Module. Initial module guides cover I2C, SPI, ADC, RTC, CAN and other modules. You will find the Module Guides collect everything you need to know to select, configure and develop with the target module. An application project ‘walk thru’ describes what the code does and helps understand how to incorporate it into a custom design.
You can find the first 10 Module Guides here. You can also use the convenient search function specifying “Module Guide” and then filtering on “Sample Programs” like this.