It depends on what role the HF has in your environment.
From a architectural point of view, the HF is usually placed in front of indexers for the following reasons:
1) Parsing - to offset parsing/typing load from the indexers
2) Segregation - in large distributed environments, indexers are frequently in more secure zones that the UF's cannot talk to directly. HF's act as gateways, if you will, to the indexers.
3) Workflow requirements - send data streams to specific indexers, or adding index time data based on location
4) Input requirements. E.g., DBX, eStreamer, and other apps that require an HF.
If you don't have these requirements, then going from a UF directly to the Indexer is not a problem. In fact, depending on the type of input you have, this should actually be faster for indexing.