It is called the long s¹ and when you look at old written documents, it seems that people did use the letter F instead of S in many words as this extract from the Parliametary Register, from 1800, shows:

If you examine it carefully, you will notice that there seems to be no real distinction between the lowercase f and the lowercase s (easiest to see in the word first):

However, you will notice that the capital letter S in Switzerland looks normal, as does the ending lowercase s in success.

The horizontal stroke in the long s, only extends to the left of the letter. In the letter f, it extends both to the left and right.

Writing has evolved over a long time. The long s derived from the Roman cursive S:

The long s was used at the beginning of non-capitalized words and in the middle of words. If s was the last letter in a word, then it was written as we write it now.By the beginning of the 19^th century, the long s began being replaced with the form we now use. Why did it die out? Probably because it didn’t make sense to have two different forms for the same letter and sound. As well, one of those forms was easily confused with letter f. Finally, no other letter in the English language had two different forms – aside from uppercase and lowercase.

A stylized form of the long s is still used in mathematics as the symbol for integration: