Wow look at this fat cat over here who can afford to lose crumbs

Depending on your hardware you should be able to have Handbrake use your GPU to reencode the video much faster than your CPU. If you have Nvidia it would be Nvenc, Intel is Quicksync, and AMD is VCE. If you select one of those as your codec it should go much faster. Check the hardware encoders section on the Handbrake documents https://handbrake.fr/docs/en/1.4.0/technical/video-nvenc.html . Even if you were using windows you would run into the same problem at some point you are limited by how fast your hardware can process the video and no software can make up for that.