To look up words in a dictionary, you need to understand the guide provided. The guide explains that dictionary entries show the English word, its phonetic spelling, part of speech, gender if a noun, and the matching French translation. It provides an example using "cat" to demonstrate how the entry is structured and labeled.