• State Suggested Answer
  • Replies 3 replies
  • Answers 1 answer
  • Subscribers 585 subscribers
  • Views 153 views
  • Users 0 members are here
Options
Related

Live object detection on RZ/A3UL using TensorFlow Lite Micro

niculaandrei

We are trying to run a machine learning model on a custom RZ/A3UL based board that captures a frame from the connected camera and detect the objects in it. In order to accomplish this we first created 2 different samples. A sample that runs an object detection model using TensorFlow Lite Micro taking as input hard coded images and a sample that can take the output from the camera using the CRU stack. These samples work as expected. When trying to combine them into a single project the camera capture buffer doesn't update. This seems like a memory problem but we don't know how to begin debugging it. Any pointers are greatly appreciated. 

  • 0

    Hii  ,

    are you referring any example projects from Renesas ? 

  • 0 in reply to GN_Renesas

    Not exactly. We took the sample available at tflite-micro-renesas for the EK-RA6M4 and modified to run on the RZ/A3UL. Instead of the sine wave approximation model we are running a face recognition model and it works great. We also made a sample for capturing images using the Raspberry Pi Camera Rev 1.3. This was done using the CRU stack and it also works great. A part of the image capturing code involves waiting for the CRU buffer to get populated with the image:

    do{
    cru_get_buffer (&capture_bufadr);
}while(capture_bufadr == NULL);

    The cru_get_buffer function gets expanded to this:
    fsp_err_t cru_get_buffer (uint8_t **buff)
{
    fsp_err_t err   = FSP_SUCCESS;

    if(cap_read_num != cap_write_num)
    {
        *buff = g_cru0.p_cfg->buffer_cfg.pp_buffer[cap_read_num];
        cap_read_num++;
        if(cap_read_num >= g_cru0.p_cfg->buffer_cfg.num_buffers)
        {
            cap_read_num = 0;
        }
    }
    else
    {
        *buff = NULL;
        err = FSP_ERR_NOT_FOUND;
    }
    return err;
}

    The cap_write_num variable gets updated inside the callback function:
    void cru_callback(camera_callback_args_t * p_args)
{
    if( p_args->event == CRU_INTERRUPT_ENABLE_FRAME_END )
    {
        cap_write_num++;
        if(cap_write_num >= g_cru0_cfg.buffer_cfg.num_buffers)
        {
            cap_write_num = 0;
        }
    }
}

    Waiting for the buffer works fine in our standalone sample but becomes an issue when trying to combine both of these samples into one application. When we are trying to run the TFLite micro model using the captured images from the camera it gets stuck in the aforementioned while loop as if the image is never saved into memory.

  • 0 in reply to niculaandrei

    Hii  ,

    Thanks for the detailed explanation. Since both your TFLite model and CRU‑based camera capture work individually, the issue during integration likely relates to memory usage or data synchronization, so to help us support you better, could you clarify whether you’re using a single .tflite model for both static and live camera inputs or two separate models (and if two, are they run sequentially rather than in parallel), where exactly the CRU buffer and TFLite tensor arena are allocated (for example, in SDRAM), and whether there’s any chance those regions overlap or are exhausting memory; additionally, have you implemented cache maintenance (invalidate/clean) before reading the CRU buffer for inference, does your cap_write_num callback still trigger in the combined project, and are camera capture and inference handled within the same loop or in separate threads or tasks?

    Kind Regards,

    GN_Renesas