Guys, please take seriously my request about thinking more open-mindedly. You're trusting the thoughts that first pop up in your mind way too much as if they were the entire picture.
(On a more general note, the world would be a much better place if everyone studied a little psychology about e.g. the cognitive biases and how much stronger they are than people usually realize, so that people would know when to stop and think instead of trusting their intuitions).
Memory-mapped files aren't some drop-in replacement for your conventional file I/O that will magically make things perform better, or make your code simpler. There is a proper way to use them for which you have to specifically design. Any contrived test case that tries to compare them apples-to-apples to conventional I/O is incorrectly designed, and will lead to these misguided metrics and conclusions.
I would advise anyone reading this thread to do their own research and learn about how and when memory-mapped files are useful.
This thread was never meant to be a general comparison about memory mapped file I/O vs conventional I/O. You guys apparently assumed it was, and turned it into that. It was originally about the surprising performance penalty when reading cached memory mapped files (and only about that). I think this is quite clear in the conclusions section I wrote, and it specifically mentions the situations where the problem might be an actual problem.
I agree 100% it's better people test these things themselves rather than trust any of the information written in this thread. But people need to be careful to make the correct conclusions from the tests, because some people posting on this thread generally have not. E.g. don't assume that if reading is slower in one case than another then it must be reading from the disk - you must think about the numbers whether they make sense, e.g. in this case the slower read was still much too fast to be the disk. And don't assume it must be copying/zeroing memory - look at the numbers to note that it's too slow to be just that. Don't assume the on-disk cache might be caching it, if it's much too small.
It is true that you can access memory mapped files from different threads once you obtained a pointer and also can request different virtual pages from different threads, if this would not work it would be afatal failure in the case of multithreaded applications. What you have to do is to manage the aquisition by your own, this is a fact you should not ignore and/or mention, especially for thos epeople reading this and don't have experience with threads.
If you try to have different threads to aquire the same page due to an API call, this will potentially cause undesired behavior so you should always take responsablity for this by your own.
By "single-threaded", I wasn't referring to thread-safety of the API, I meant that (according to some sources) the code inside Windows that handles the page faults and the update of page tables is single-threaded and does not scale to multiple threads (which I haven't verified myself).
But this is true as long as you share the memory mapped file on both instances of the process. If you don't share, Windows is creating two different memory mapped file instances with double amount of pages etc.
No, that does not seem to be the case. I verified this again now and even if using the same CreateFile() and CreateFileMapping() handle, just opening two views with MapViewOfFile() will be equally slow (the first time each view is created that is, the second read is of course fast after the pages have already been mapped).
ptr1: 00000291E3D20000
read 1: 2.48832 GB/s
read 1: 24.1804 GB/s
read 1: 23.4896 GB/s
read 1: 24.1995 GB/s
read 1: 23.4662 GB/s
ptr2: 00000292D9EC0000
read 2: 2.64585 GB/s
read 2: 23.7152 GB/s
read 2: 22.0792 GB/s
read 2: 23.5864 GB/s
read 2: 22.0855 GB/s
This is also not true because if you close all processes that hold a specific mmf, it will be removed from memory at all because no one uses it again. Why should Windows keep them cached, this is bad design in best case but a potential security issue at all.
Windows uses unallocated memory of the system to cache the contents of recently-accessed files. The reason is that it makes future reads faster, just like any other cache. This is also very common knowledge, I'm surprised if you didn't know this. And how on earth is this cache "bad design"? It's not really considered a "security issue" either. Perhaps you're thinking about something else here, but I don't know what. Anyway my original point and evidence is clear.
Whether Windows caches files when reading them by memory mapping or just when using conventional I/O was another question, but like I said a couple of times already, I verified this and indeed it does. And of course it should, why would it be different than conventional file access.
But now, I've used a lot of time thinking and testing and verifying these things, and every time someone claims something, I've been thinking my head off whether there could be something there that I'm missing, and tried to verify everything, even though everything seems to point to the opposite. At the same time, you guys are just posting whatever pops up in your mind without testing anything, or bothering to think if your conclusions are even logical in the first place. Unless there is some seriously new information, preferably with backing test results or other credible verification, I must conclude that all the relevant information is still in the summary I wrote, and most of everything else is just unrelated or downright misinformation.