AutoCad :: PDF With High Resolution Image Huge File Size

I am trying to export PDFs of drawings with a high resolution image covering almost an entire A1. I realize of course that the file will be very large but it is much larger than I feel it should, and there are also some discrepancies.

The original image is around 100 MB, but only about half is visible in the viewport, so the PDF should be around 50 MB if there is no compression (only negligible amounts of text and lines besides the image), however if I plot it to a PDF using Adobe PDF, setting the resolution to 300 dpi and medium JPEG compression the resulting file is 150 MB. If I choose 72 dpi and low JPEG compression the file looks very bad and compressed but is still around 40 MB in size. This is far too large considering the amount of compression visible in the image, it looks like a 500 KB file tops.

However if I use the DWG to PDF printer instead of the Adobe PDF I can get a very good looking file at around 2,5 MB. Very suitable for printing, but the compression shows if you look up close in the computer (which is a requirement for these files). The odd thing here is that it makes no difference what resolution I set it to, 150 DPI to 1200 DPI creates the same looking file at exactly the same size. I'd like to be able to create a 2,5 MB file to email and a very high quality file for archivation, but not 150 MB large.

Another odd thing is that if I print another file but with the same type of image (a different facade) with 1200 DPI I get a very high quality file of 40 MB, but if I repeat the exact steps and settings for the first file I end up with a 150 MB file. Of course some differences in the image would account for some of this, but not nearly of this magnitude. I deleting everything from the large file and copying in the content from the small file, and successfully printed a 40 MB file again, so there doesn't seem to be some setting or bug in the file I am printing from, rather something in the original file with the image that I have Xref'ed.

How I would go about creating a very high quality file that doesn't bloat up way more than the original image? A 40 MB file is great considering the images are huge, but I can't get this result consistently.