Why is hydrogen less electronegative than Fluorine?

If the trend is electronegativity increases from the left to the right of the periodic table, and also from the bottom to the top, then why is hydrogen not very electronegative. It only has one electron, so it would tend to hold onto it right? If we move hydrogen over to the halogen section (instead of seeing it as only one electron, this would be viewing it as needing only one more electron to make a full valence shell) then it would appear to follow both of the trends I stated above. Can someone explain this?