Converting EPUB to PDF using Pandoc

Have you tried the latest version of Pandoc, which currently is v2.5?
Have you tried to play with '--epub-chapter-level=...' numbers?
Have you tried to play with '--top-level-division=section' (or ...=part or ...=chapter) for the output?
Have you tried with the LuaLaTeX engine`?

pandoc      \
  -f epub   \
  -t latex  \
  -o my.pdf \
  --epub-chapter-level=1 \
  --pdf-engine=lualatex  \
   test.epub 

Have you tried a non-Pandoc tool like Calibre?