The short answer is yes, in theory this is very much possible.
It is also one of the many reasons you should disable java-script when using tor.
Most of the time a vulnerability in your browser gets exploited and some code will be executed in the background. Hence downloading a file over said browser isn't necessary anymore. You get infected "drive-by".
Is it likely to happen, or can you create something like this?
Probably not.
1) If your asking such a question you are not skilled enough to find such a vulnerability.
2) Such an exploit is very valuable and rare, therefore it is more and more unlikely that you encounter one in the wild targeting every user visiting a site, cause then there is a higher chance people notice and in the end it gets fixed.
but:
It is still definitely possible that people with such an exploit will run a malware campaign on some "adult video" websites or so and try to target as many machines as possible, you never know and in the end they decide what they are going to do with the exploit. Its just my opinion that this is the more unlikely scenario.
tldr: it is possible, stay safe, dont be stupid. And being honest: in the end you really can't do much against it