MCUBoot and RA6M4

I am using the EK RA6M4 and have tested the application note R11AN0570 to use the MCUBoot with the dual flash bank. In the application note the UART is used to transfer a secondary image to the EK using the XModem protocol. I am able to update the firmware over the uart. A small tweak is that I am not using the RTOS but a single main loop.

Now we have created a custom PCB using the Renesas R7FA6M4AF3CFB and I am testing the same application note on the new PCB but this fails. I am reusing the bootloader project (ra_mcuboot_ra6m4_dualbank) and running the app_primary_uart binary. After connecting and transfering the app_secondary_uart binary, the debugger is indicating an error. The microcontroller is crashing after the system reset after swapping the images. It can not start up with the secondary image.

So why is this project running on the EK RA6M4 but not on my custom PCB with the R7FA6M4AF3CFB microcontroller?

The pimary application was running and I started a file transfer of the secondary application. This goes well until the point of the function R_FLASH_HP_BankSwap in RM_MCUBOOT_PORT_BootApp. After the call to __NVIC_SystemReset();, the debugger is jumping to break at address 0xfffffffe

These are the regions for the dual bank images.

#define PRIMARY_IMAGE_START_ADDRESS 0x00020000
#define SECONDARY_IMAGE_START_ADDRESS 0x0220000

If I look at the start address of the secondary image, I see that it is located there after the file transfer over the uart. The version number is 1.1.0 

Does anybody has an idea why the microcontroller can not boot with the secondary image?