Comparison of Trajectory Programs

Dave Tutelman - January 29, 2007

This mini-study was provoked by response to my study of launch space surfaces, and my driver fitting recommendations based on that study. The accuracy of the Tom Wishon Trajectory Profiler was called into question. David Bahr in particular suggested that it does not do as good a good job with drivers as it may with irons.

This motivated me to take a few known real-world points in launch space and see how each of several programs stack up. The programs are:
WishonTom Wishon Trajectory Profiler version 2.0.
Nice GUI on a commercially available program. Has separate modes for launch parameters directly or for impact parameters (which computes launch parameters). This dual-mode interface made it the first choice for this study. It can be ordered from Wishon Golf's web site.
Actually, this software was develeoped by GolfMechanix and "private labeled" for Wishon. The same software is available directly from GolfMechanix.
DupilkaNiblick Golf Crafter Trajectory Model version 3.0.
GUI similar to Wishon, and predates it by years. Only has a single mode: impact parameters. In order to use launch parameters, you have to guess a good set of impact parameters and tweak them until it shows the launch parameters you want. This software used to be freely downloadable from Max Dupilka's web site, but he has discontinued it.
AdamsJohn C. Adams' Golfball Spreadsheet version 2D.
This is an Excel spreadsheet that Adams used in some research, presented to the American Society of Mechanical engineers 2/20/2004. You can see his excellent presentation and download earlier versions of his spreadsheet at the ASME site. This program was intended for driver-type trajectories only; it gives truly silly results for short irons, and Adams will be the first to admit this.
OptimalFlightTodd Kos' OptimalFlight.
I do not have this software yet, because it requires a later version of Excel than I have on my computer. Todd provided me with its output for the sample launch parameters, so I could include it in the comparison. You can see more about the program at Todd's web site.
Here is the comparison table. Points to note:
  • The "Real World" row contains actual, measured carry distances. They are the accuracy standard by which the programs need to be judged.
  • All these points involve driver lofts and ball speeds. This study was not interested in how well the programs simulate irons, just drivers.

Long
Drive
1
Long
Drive
2
Big Tour
Drive
Robot
1
Robot
2
Bob
1
Robot
3
Robot
4
Bob
2
Launch
Parameters
Ball Speed210197185154137133130123121
Launch Angle7.67.315.511.711.714.412.614.118.2
Spin320029001800350024503270310027002350
Distance /
Hang time
Real World362336~320252214224203186212
Wishon323 / 9.4310 / 8.8284 / 8.7244 / 7.8217 / 6.5226 / 6.8208 / 6.6196 / 6.2204 / 6.4
Dupilka350 / 8.2327 / 7.4321 / 7.7265 / 7.7226 / 5.9235 / 6.7220 / 6.4205 / 6.0214 / 6.1
Adams358 / 8.3333 / 7.4317 / 7.7250 / 7.0212 / 5.4220 / 6.0202 / 5.7189 / 5.3197 / 5.4
OptimalFlight360 / 8.4336 / 7.5322 / 7.6253 / 7.0215 / 5.4222 / 6.1205 / 5.7191 / 5.3199 / 5.5
Impact
Parameters
(needed for
Dupilka)
Clubhead Speed14013412410693.591.589.38482.5
Loft7.97.75.211.79.312.712.311.410.1
Angle of Attack0.60.410.81.63.53.52.04.29.4

Results

Yardage error

The most important result is the difference between the program's output and reality. So the first thing I looked at was the yardage error. I plotted the error against ball speed, launch angle, and spin. The only one where there was a visible correlation was ball speed. Here is the graph of yardage error vs ball speed.



My observations:
  1. The sample at 121mph is probably a wild data point. All the programs estimated its distance about 17 yards lower than that program's estimate for nearby samples. I'll ignore that sample in the subsequent observations.
  2. The Wishon and Dupilka programs had a substantial correlation between ball speed and error. The signed value of the error decreased with increasing ball speed. In more detail:
    • The Wishon program was at its most accurate at about 140mph of ball speed, while the Dupilka program was at its most accurate at about 170mph of ball speed.
    • At long-drive ball speeds (over 170mph), the Wishon program was way low -- 25- 40 yards off. The Dupilka program gave usable results, within 10 yards.
    • At typical golfer ball speeds (120-150mph), the Wishon program gave usable results, within 10 yards. The Dupilka program was 10-20 yards high.
  3. The Adams and OptimalFlight programs did well -- inside 5 yards -- for all ball samples except the outlier at 121mph. OptimalFlight usually did a little better, but both gave better than merely "usable" results.

Shape of the launch space

Critical to my work with optimizing drivers is the "shape of the launch space" -- that is, the shape of the surface when you plot yardage against both launch angle and spin for a given ball speed. Here are two very different "shapes" for a ball speed of 124mph (corresponding to a clubhead speed of about 85mph).

Wishon - The Wishon program produces a launch space that looks like this. It is a chart of carry distance in yards for combinations of launch angle and spin.

In order to visually enhance what the graph looks like, I have added color representing the yardage. The redder the color, the longer the drive. The colors go through the spectrum, all the way to violet for short drives.

Looking at this chart, we can see:
  • The maximum drive for a ball speed of 124mph is 201 yards. The conditions to attain this drive are 20º of launch angle and 2500rpm of spin. (There is a red circle around the maximum.)
  • The 201-yard point lies on a "ridge" of fairly long distances going diagonally across the chart. The ridge is represented by the black dotted line. Distance decreases in both directions from the maximum, but very slowly.
Adams, Dupilka - This launch space was produced by the Adams spreadsheet, but the Dupilka program creates one very much like it. It is similar to the Wishon launch space in a few ways, but distinctly different also:
  • First, a similarity. At 20º and 2500rpm, the distance is still 201 yards.
  • Another similarity. There is a diagonal ridge of good drive distances.
  • Big difference! 201yd is not the maximum on this graph. Distance continues to increase as you go to higher launch angle and less spin. We have only plotted it to 26º, but it is not showing signs of decreasing again, the way the Wishon plot did.
  • Another difference, and an important one. As we go left along the ridge (to lower launch angles and higher spin), the distance drops off significantly -- much faster than on the Wishon graph. For instance:
    • On the Wishon plot, moving 6º along the ridge, from 20º to 14º, you lose only 2 yards of distance (201 to 199).
    • On the Adams plot, moving the same 6º along the ridge, you lose 11 yards (201 to 190).
  • Not only does the ridge have more variation as you move along it; it is also "sharper". That is, you lose distance faster as you move away from the ridge. This is very visible from the much larger violet areas (lower left and top right) on the Adams chart than the Wishon chart.
Those are big differences. And those same differences appeared in launch spaces at 100mph and at 200mph ball speeds as well.

Other Observations

The Wishon program differed from the other three in a few additional ways:
  • For ball speeds above about 135mph, the Wishon program gave significantly higher hang time estimates for the ball flight. For lower ball speeds, Dupilka gave almost as large a hang time as Wishon, and both significantly higher than Adams and OptimalFlight.
  • I made it a point to observe the peak height of the trajectory for some of the samples. The Adams, OptimalFlight, and Dupilka programs had very similar heights, but the Wishon program predicted that the ball flew significantly higher, usually about 20%. This probably strongly related to the higher hang time.

Conclusions

I no longer trust the Wishon software for variational analyses. I never really trusted it for absolute distances, but used it a lot for observing trends when varying parameters. After this data, I can't trust it with trends either, because:
  • It has a distinct bias with respect to ball speed, so any trend that involves ball speed (directly or indirectly) is suspect.
  • It gives a different shape of launch space from that of the other programs. This fact is true at several ball speeds, including some where the Wishon program clearly does not predict distance well. So I trust the other programs more. It would seem that the Wishon software has other biases beyond just ball speed, because ball speed was held constant in each of these plots of launch space.
  • Note that there is no data on ball speeds below 120mph. That is a shortcoming of the study. It is possible that the Wishon program might actually be more accurate than the others for low ball speeds. There is no trend to indicate that this is true, but it can't be ruled out completely yet.

Next Steps

The first of the "next steps" has been to build a program that:
  • Gives similar results to those of the OptimalFlight and the Adams spreadsheets.
  • Has a flexible and powerful user interface -- a lot more so than can be accomplished by layering an "application" on top of Excel.
I didn't realize that was the next step until Frank Schmidberger contacted me, having seen a draft of this report, and proposed it. We collaborated to produce TrajectoWare Drive, which you can download for free from http://www.trajectoware.com. If you're interested in the subject matter of this report, you will want to download and try it. Feedback will be welcome; please leave it on the forum, so others can discuss it as well.

I need to re-work and re-write my article on optimizing a driver, because I no longer trust the Wishon software. I will use the TrajectoWare Drive this time. I expect the difference in the shape of the launch space may produce some differences in the strategy I recommend.

There may be figures in the Club Design Notes that also need to be re-worked, though I don't think the verbiage that goes along with them will require much change.

Finally, if anyone can give me reliable data for lower ball speeds, I would much appreciate it -- and will do the work to see how it fits into the overall conclusions. "Reliable data" means:
  • Measured ball speed, spin, and launch angle.
  • Measured carry distance (total distance, too, if you have it). I am not interested in the data if the distance was computed by the launch monitor rather than actually measured. But the measurements don't have to be "paced off"; they can be radar or laser measurements, such as Trackman supplies.
  • Other environmental info, such as altitude (if not at sea level), wind speed and direction, and temperature.
  • Preferably several (4-8) similar hits, so wild data points can be identified and ignored.
Thanks in advance.

Appendices:

1. Information about drive data

The source for this data is Todd Kos, except where noted otherwise.

Long Drive 1Jason Zuback, drive #1 at ReMax finals, 2006.
Long Drive 2Erik Lastowka, Drive #4 at ReMax finals, 2006.
Big Tour DriveReported drive of Bubba Watson. Distance measurement not exact.
Robot 1Golf Labs robot test, made for Golf Digest research article.
Robot 2-4Robot test data - averages - from various sources (GolfWorld, etc.)
Bob 1-2FlightScope Pro measurements in Green Valley, AZ (2700 ft altitude). Source: Bob Barrette.

2. Settings for the various programs

I used the default conditions for most of the things in each program. This is especially important for things like the aerodynamics of the ball. I simply went with the default ball for each program. Here are the exceptions to that rule of "go with the default".
  • Altitude / air density - All the programs have "sea level" as their default. Some of the data were taken at significant altitude. The way I accounted for this in the programs was:
    • Wishon: set the altitude directly with a control in the program.
    • Dupilka: set the altitude directly with a control in the program. Dupilka's control also shows the fractional pressure density, compared to sea level. I copied this number down.
    • Adams: this program takes barometric pressure in inches of mercury. I used the fraction copied from the Dupilka program, and multiplied it by the default pressure from the Adams program. I plugged the result back into the air pressure box in the Adams program.
  • Other atmospheric conditions - All programs allow adjustment for temperature. It was set the same for all programs. While it might not have been exactly as the conditions for the real-world strike, (a) none of the strikes were under extreme conditions, so it would not matter much, and (b) it should affect all the programs the same in the error induced. No wind was reported for any of the strikes, so none was cranked into those programs that deal with wind. That may have been oversight on the part of the person reporting the data, but it is the best we can do with the data. However, it might account for the one wild data point, where all programs reported a considerably lower-than-reality carry distance.
  • Launch conditions - All the programs except Dupilka start with launch conditions (ball speed, launch angle, and spin). Dupilka only allows impact conditions (clubhead speed, loft angle, attack angle, clubhead mass). It does provide launch conditions as output. So I "tweaked" the Dupilka program for each data point, to get it so the input impact conditions resulted in the desired launch conditions.
  • Other default parameters:
    • Adams has a tee height that defaults to one yard. I set it to zero, to match what the other programs do.
    • Dupilka, having been written in 1999, has a COR of less than the currently-allowed 0.83. I increased it to 0.83 using a control provided in the program. This is really a non-issue anyway, since it is an impact condition; as long as the launch conditions were tweaked to be right, it doesn't matter what the COR is.


Last modified  July 21, 2007