The fuel entries menu can be used to configure which items count as fuel in a given fuel registry. It is basically a list of items with associated durations (how long a container burns when this item is used as fuel). If you have a single entry that allows coal to keep the container burning for 5 seconds, it should look like this:

Furthermore, there are a couple of components for every entry that is currently in the entry list: