Definitely worth doing once if you like exotic motor vehicles. It is quite hard to see much of the hillclimb unless you have a seat in one of the small grandstands - not many viewing places elsewhere. We ended up standing under the trees down by the start, but the cars weren't really travelling at that point so it wasn't ideal. If you're not especially bothered about the hillclimb there is plenty to see, for example the Style et Luxe display or just wandering round the paddocks. It is all a bit unreal - true exotica which are way beyond the means of normal mortals - but nice to do at least once, on a nice day. It is very well organised, plenty of parking in the fields below the Festival site, and you can get close enough to the cars to take some great photos.
Once you've done the FOS I'd recommend something a bit more down to earth like the VSCC race series, especially the one at Mallory Park in August, when they have a race for Edwardian vehicles. Quite mad, and some of the drivers are great characters, and it is much cheaper to attend than the FOS. The NSRA Nostalgia Nationals near Stratford at the end of June is fun too!