This essay will be a walk down memory lane in which I will try to explain the significance of the article "Quantum-chromodynamic gluon contributions to large-pT reactions", of which I was a co-author with Dennis Sivers. This paper has been cited a large number of times (415 at last count, which in this field is a lot), which means that it had a significant influence. Here I will try to put this work into the perspective of the time, as best as my sometimes fuzzy memory allows. This was, after all, more than forty years ago -- but times were different than and I'll try to explain how and why.
At the time this work was done the idea that hadrons (baryons and mesons) were composed of constituent quarks bound together by gluon interactions was becoming quite popular for a variety of reasons. First, and most basic, was that the theory of how quarks and gluons interact was a gauge theory. Gauge theories have long been very popular in physics because they are derived from fundamental symmetries. At the time it was certainly known that the U(1) gauge theory known as Quantum Electrodynamics (QED) was one of the most successful in the history of physics, making many confirmed predictions including the astounding precision of its prediction of the electron magnetic moment. QCD (Quantum Chromodyamics), the gauge theory for quarks and gluons, is based on the SU(3) symmetry.
Second, the idea that hadrons are composed of quarks had been quite successful at explaining hadron spectroscopy -- the otherwise bewildering multitude of mesons and baryons showing up in scattering experiments -- as various combinations of the different types of quarks.
Third, it was at the time well understood why free quarks and gluons were not observed. In 1973 Gross, Wilczek, and (independently) Polizer discovered that QCD is "asymptotically free", which means that the interaction between quarks and gluons becomes weaker at small distances but stronger at large, making it impossible for a quark or gluon to escape its binding in a hadron.
And finally, it had become apparent from deep inelastic scattering from the proton that there were small, hard things inside. This was directly analogous to the discovery of the atomic nucleus in 1911 by Rutherford. In that case they observed astoundingly large angle scattering of alpha particles from a thin gold foil. This made it clear that most of the mass and all the positive charge of the atom were concentrated into a very tiny (compared to the atom itself) nucleus. Similarly, large angle scattering from protons made it clear that something small and hard was inside. It was very attractive to identify these small things with quarks, and many physicists followed that road, but there was a big difficulty: a simple one-gluon exchange calculation of the scattering cross section for large angles predicted a falloff considerably different from that observed experimentally. In fact, it appeared that it predicted a different power law (pT-4 rather than pT-8 as approximately seen in the data) for the decay with transverse momentum (that's the pT in the title of the article). This fact led Feynman, at the time certainly the world's pre-eminent physicist, to resist strongly the identification of the hard seeds with quarks and to refer to them only as "partons".
This was the situation at the time, and what we tried to do was resolve the problem with the large angle scattering by attempting a QCD calculation that went beyond the leading order one-gluon exchange to diagrams involving two virtual particles. This turned into a very complex task for a variety of reasons, discussed in stunning detail in the article, but one of these reasons was that these diagrams, and there are a bunch of them including gluon-gluon scattering contributions, expand into a very large number of terms that must be algebraically combined. We're not talking anything exotic once one gets to the polynomials having worked through a lot of other problems, but these polynomials can be very large indeed. In fact, a direct evaluation of the gluon-gluon diagram results in 228,420 terms, which is obviously quite daunting. In order to evaluate most of these large expressions I borrowed an algebraic-manipulation program called ASHMEDAI from Michael Levine, a physicist friend at Carnegie-Mellon who developed it to evaluate the high-order QED diagrams involved with precision predictions of the electron magnetic moment. This type of computer program is very common now, but at the time ASHMEDAI was pretty much unique. Unfortunately, 228,420 terms was a bit much even for ASHMEDAI, so I turned to a different technique for this diagram which involved introducing the so-called Fadeev-Popov ghost. I've switched to the first person here because I was the one who brought in the Fadeev Popov ghost technique, and as you will see there may have been a problem here. However, to continue the narrative, we hammered through the calculation and derived results which removed some but not all of the discrepancy between prediction and experiment. The trend was, however, in the right direction and it was enough to give one hope that if one continued to even higher order diagrams (and a very, very daunting task that would be) the entire discrepancy would go away. At least, that was the hope, and from the number of citations this paper received it seems that the idea was reasonably convincing. In fact, it is now generally accepted by the physics community that the "partons" are in fact quarks, although I'm not exactly sure how that consensus arrived because I got out of the field into the oil business shortly thereafter. My suspicion, however, is that our results were plausible enough, and the possibility of expanding them to higher orders so difficult, that most people figured that the identification as quarks was probably OK and not going to get any better from that viewpoint.
Below are figures showing in more detail the results of this calculation.
The first, with no label, shows as dashed lines the result of calculating
one-gluon-exchange compared to data (from an earlier article by Sivers and
myself, "Quantum-chromodynamic gluon contributions to large-pT
reactions"). The range of dashed curves attempts to quantify the
uncertainties in the calculation. It is easy to see that the one-gluon exchange
prediction has a different falloff behavior with pT than the data as well as being systematically smaller.
Now let's look at similar plots of the predictions from the second order
calculation described above. The two figures correspond to two plausible
assumptions for the shape of the gluonic content of the proton, and again the
double dashed curves represent an estimate of the uncertainties in the
calculation. It is clear that the agreement with data, although not
perfect, has improved both in shape and size.
Oh, I said something about an error. After we published a preprint of the paper another group calculated the gluon-gluon diagram, using a completely different technique that I didn't really understand very well, and got a different answer. They claimed that their result was demonstrably gauge invariant, whereas I could only hope that the Fadeev Popov ghost, originally introduced specifically to maintain gauge invariance, actually did its job. So we added a "Note Added in Proof" reading (slightly edited),
It has come to our attention that the calculation of Combridge, Kripfganz, and Ranff for [gluon-gluon scattering] differs from ours by a small term .... They use explicit polarization vectors instead of the ghost-subtraction procedure described in Sec. HI. We suspect their calculation is correct, but we have not been able to find any error in ours. We thank B. Combridge for communicating their results to us prior to publication. None of our numerical results are sensitive to this [since the difference is a small term].
One of the things my graduate advisor taught me was if, even after the most meticulous checking and re-checking, one ends up making a mistake -- one needs to own up to it. I learned much more than just physics from H.W. (Bill) Wyld.