What are we trying to do?

Basically we want to find some nice starting words for wordle, but thats pretty easy to do so we would also like to know a few ways to eliminate as many characters as possible from the board. Also if possible rank these different ways according to whats most probably beneficial (I basically know nothing about statistics so this ones a big maybe).

Character Frequencies

We could probably find some data file somewhere that has a good metric for char frequencies, but that would be cheating, so were gonna generate them ourselves using War and Peace from project Gutenberg.

This is what the getCharFreqs() function does for us. Graphing it we get

This does not match exactly with other sources, but its close enough for our purposes. We’re not trying to break some encryption here, just get a decent rating system.

5 Letter english words

We need a word list of english. We’ll grab words.txt from https://github.com/dwyl/english-words/.

In the interest of eliminating as many characters as possible we want only 5 letter words that have unique characters. The word wells wouldn’t be a good guess for example because we only eliminate a max of 4 chars instead of 5.

We also don’t really care about words that have the same letter as some word already in our collection. In otherwords we don’t want any anagrams, they’re not helpful.

Also our wordlist seems to contain a lot of crap that does not spellcheck, so we’re gonna use the enchant python library to get rid of anything that doesn’t spell check.

After we’ve filtered out everything we don’t care about we’re left with some 1765 words. These can be found in 5_letter_words_no_2_have_same_letters.txt

Ranking them according to the frequencies we found before we can see the top 20.

Mutually Exclusive Sequences

We want a sequence of words that have no letters in common between them. If we could find 5 of these we could eliminate 25 words from the board and our 6th guess would be pretty much guaranteed. Except for the fact that you may get a bunch of letters out of order then your kinda screwed and your last guess is basically a hail mary. In reality I’m not sure that there actually is a sequence of 5 such words, but we can find a bunch of sequences of 4 words, and some smaller sequences of 3 and 2.

This is what the wordGroups() function does for us, it basically just takes all the words we found in the previous step and finds all mutually exclusive sequences.

[swamp befit chord gunky] is of the sequences the rest can be seen in mutually_exclusive_words.txt

Rating Our Seqeunces

The simplest way to order them would weighting each word by char frequency and summing the weights, but thats not super useful, since if you choose a large seqeunce your gonna eliminate most of the chars anyway.

It would be nice if we could rate them in a way such that the words in the sequence contained frequently occuring characters in a sorted order. So we could elimnate the 5 most common with the 1st word, the next 5 with the 2nd, and so on. However I am bad at statistics, sooooo, we’re just gonna sort the sequence by char frequency ascending, weight the words based on index, and sum them. Hopefully we promote sequences of the kind we’re looking for by weighting things in this way, but it could also be pointless. At any rate its not worse, than the basic metric.

This is all accomplished by the orderLists() function.

Ordering Sequences by Length

As you would probably expect, there is no sequence of 4 words that has a lower weight than a sequence of 3, or 2 words. The same is true for sequences of 3 and 2 words, so lets split them up and put them in their own file. Most likely your going to want seqeunces of different length anyways. If you really just yolo with a sequece of 4 everyday your kinda nuts.

This what writeWordLists() does and it produces the following files.

• group_0.txt sequences of length 4
• group_1.txt sequences of length 3
• group_2.txt sequences of length 2

These are the first 20 entries of the sequence tables

Sequences of Length 4

Sequences of Length 3

Sequences of Length 2

If you want to look at all the code the main file that does all of this is freqeuncy.py its really gross code though so… have fun?

Good luck with wordle!!