I've been using Math.random() in JavaScript (both in the browser and on the server) to generate random numbers for a couple of years and, for reasons I can't explain, I get a lot of duplicates.

Yes, I know, it's random. So you will get duplicates. But I was getting duplicates on the order of every 10th number, when selecting from a range of over 1000 possibilities. And this happened every time I looked at a sequence that long.

I Googled it exhaustively and found nothing meaningful.

It is true that Math.random() is not really random of course. It is pseudo-random - computers such as you and I have on our desks, or such as you can get provisioned (easily) in cloud data centers, cannot generate real random numbers.

But when you look at the articles on these pseoudo-random number generators (PRNGs) you see that it should not happen as often as it was happening to me.

Here are some useful articles on how they work:

https://hackernoon.com/how-does-javascripts-math-random-generate-random-numbers-ef0de6a20131

https://v8.dev/blog/math-random

They're a little techy, but so is this topic.

Today browsers offer Crypto.getRandomValues(), which is a good deal more secure, but I simply decided that I wasn't leaving this to pseudo-random chance, I went looking for where I could get real random numbers on the internet and I found two places.

The first I already knew about, random.org. It has been around for a good long while, and it has an API.

The second I found was a consortium, seemingly led by Cloudflare. I know Cloudflare as I use them on another site for DNS security, but I was unaware of their work around cryptography.

Here is a link to some information about Cloudflare's project 'League of Entropy' and also another cool project they have for generating their own (internal) random numbers from lava lamps:

League of Entropy
Lava Lamps

Using Cloudflare's solution was more complicated than what I wanted and I was not sure exactly how to adapt it for my needs, so I went with Random.org instead.

Random.org's API was very easy to use and their testing tool was very helpful. Here is a link to their request builder:

https://api.random.org/json-rpc/2/request-builder

Random.org uses atmospheric radio noise to generate real random numbers. That is, they have devices (in multiple countries) tuned between commercial radio frequencies and the background noise from these radios is used to generate a random signal that produces random numbers. Neat! Here is a link to a faq question about it that is very interesting:

https://www.random.org/faq/#Q1.4

The only thing that I did not like about it, is that I needed to round-trip to random.org to get a random number, and I had to figure out how to secure the license key. I did not want a lot of latency and I did not want to publish my license key to every browser in the world. Here is what my solution looked like:

1. In browser, generate a psuedo-random number on page load. This is a fallback if other things break.
2. In browser, make a background request to the website back-end for a real random number.
3. On the server, if I have a real random number available, send it to the browser.
4. On the server, if I don't have a real random number available, make a request for a buffer of random numbers from random.org. Generate a pseudo-random number and send it to the browser to avoid any additional latency.
5. On the server, once I receive a buffer of random numbers store it.
6. In the browser, make use of the random number.

I was a lot more worried about introducing latency than (and failures) than I was about whether or not I had a truly random number every single time, since this isn't exactly a high security system.

Nonetheless, I wanted a random number so I implemented the code above to give myself true randomness most of the time, with a fallback to pseudo-randomness to avoid excessive latency or failures. It was about 3 hours to get it all working and tested for edge cases such as "What happens if the web service fails?" or "What happens if my buffer counter for random numbers ends up in the negative for some reason?"

I still want to get to the bottom of why I was seeing so many duplicate numbers from Math.random(). I have to believe it was something to do with my code and not with Math.random() itself.

I am going to branch that effort into a different project to see if I can reproduce the behavior and get to the bottom of it.