The tool was interesting, from what I can tell relatively accurate (based on the Outcomes Of The Match stuff the NRMP puts out), and easy to understand, but I question its usefulness. The only real uses I see for it is maybe to give an M2 studying for Step 1 a rough goal to reach for a Step 1 score and just maybe a very vague guide as to how long your rank list needs to be before you can say you will match in a specialty with near-certainty. However, there are a lot of other factors that go into determining your competitiveness in the match. Here are some of the bigger ones I see:
1. Your personality. You have to fit in with the residents and overall culture at an institution for them to rank you decently.
2. Black marks on your record: failed courses or rotations, failure/retake on any of the steps, bad evals, any action for anything unprofessional, criminal record.
3. Whether or not you are couples matching. Couples matching usually results in a match slightly lower on your rank list.
4. How long your rank list is. This is nearly as important as your Step 1 score.
5. Where you went to med school.
6. For some programs, it also matters if you have any specific ties to the state where you are applying for residency.
7. Competitiveness of the residency program. You'll have a harder time matching to a better residency program versus a poorer one (duh.)
I realize the little tool can't get into all of that, but that's what your adviser is for. They can usually tell you about how competitive you are in applying to a certain specialty at places you'd actually apply to once they get to know you.
Also, somewhat OT: I think some of the tools like this can actually be harmful since some people use them to basically decide what specialty they should go into. I've heard "Oh, I got a [something 250 and above] on my Step 1, I better do derm/ortho/radiology so I don't 'waste' my score" more times that I care to recall. Also, this occasionally leads to people who do get decent Step 1 scores being told not to do specialties that don't absolutely require ridiculously-high Step 1 scores. My entire class was surprised at my specialty choice since I did pretty well on my Step 1 and primary care is apparently "what people who got a 186 on their second try do."
The Step 1 is just an expensive test of how well you are willing to sit for months and months and memorize inane little pieces of information that often have little value outside of medical research. For example, did you ever pick a benzodiazepine over a barbiturate in a clinical setting solely because they cause the Cl channels on the GABA receptor to open more frequently instead of for a longer duration like the barbiturates do? The NMBE discourages the use of the Step 1 for ranking applicants for residency programs, and it's very likely for the reason I stated above. There's even a lot of rumor about making Step 1 a pass/fail test as a result of programs relying on it to screen/stratify applicants. I can easily say that I've worked with people with 250s on their Step 1 who aren't worth a darn in a clinical setting.