Caliber of a shotgun is traditionally referred to as the gauge of a shotgun. This number indicates the number of balls matching the bore diameter that could be cast from 1 lb. of lead. Thus, the smaller the gauge of a gun, the bigger the bore. The biggest gauge legal for hunting in Colorado is 10, and the smallest found to be practical is .410 (which in fact is a caliber, or bore diameter in inches; it is slightly smaller than gauge 32).
Shotgun barrel is not uniform in diameter from chamber to muzzle. Usually, it narrows down a little just 1-2 inches before the muzzle to produce tighter shot patterns. These "bottlenecks" are called "chokes". Chokes measure only thousands of an inch smaller than bore diameter. However, given the speed that a shot load develops while traveling down the barrel, even such slight choke is capable of dramatically changing shot pattern. Chokes may be built-in, or stationary, for some guns or may be removable, so that shot pattern can be adjusted for different hunting or sport shooting situations.
For big game hunting, slugs are used in shotguns. Slug hunting probably deserves more attention than it is given, especially for short-range, brush country situations. Shotgun slugs are typically several times heavier than rifle bullets, and are made of much softer materials. This causes most of the energy of a shotgun slug to dissipate in the target on impact, literally just like a devastating blow of a sledgehammer. Rifle bullet, although having much more kinetic energy at large distances, also inflicts a lethal wound, but at short ranges just pierces the body of game and continues to carry for another mile. Dangerous animals may not be stopped this way. Of course, modern rifle bullets are designed to expand on impact, but even the expanded rifle bullet barely reaches the diameter of an unexpanded shotgun slug.
At long ranges, large cross section of a slug makes it lose energy fast. Additionally, slugs are not stabilized by the gyroscopic effect as are rifle bullets, and they tend to start flip-flopping in flight eventually. Therefore it is difficult to recommend them at ranges over 100-130 yards. Thinking about the relationship between the drag on the flying slug and its diameter it appears that smaller bore slugs should lose energy slower than large-bore slugs. However, it is still impossible to launch a slug at the same muzzle velocity as a rifle bullet because of the thinner shotgun barrels. Do not attempt to overload shotgun shells to achieve higher shooting distances.
There are ways to stabilize slugs and improve long range accuracy. Rifled barrels are the best way and are available for most shotgun models. Rifled barrels result in higher accuracy than smooth barrels. Another way is to use specially machined, hard-alloy slugs, such as the Blondo bullet, which was used by the French resistance to shatter the Nazi's engine blocks during WWII. Machined from brass, the Blondo does not deform in the barrel as any lead slug does, leads to higher chamber pressures and should not be loaded with excessive powder charge. The rigid bullet utilizes the turbulence behind its head to decelerate the skirt, thus staying head forward in flight. It does not deflect or deform when hitting large bones but shatters them, keeping going straight. Not having used it myself I read the reports of the Blondo being more effective on large game than lead slugs, delivering deeper wounds than lead slugs. The best way to use a Blondo is with a two-piece sabot.
Please note that the following discussion of buckshot is mostly of theoretical interest because buckshot is not allowed for hunting use in most jurisdictions.
Buckshot. Shotgun loads don't have to be limited to shot (small size pellets made of lead alloys or iron alloys). There is definitely at least another load that is widespread and very practical, although no longer widely used in hunting applications. It is buckshot.
You'd have to wonder, why do police officers and other law enforcement agencies use buckshot instead of slugs, or prefer smooth-bored shotguns to rifles? There have to be several reasons, one being, they say, the psychological effect of the sound of the cycled shotgun slide. However, there are more practical advantages.
Multiple smaller wounds produced by buckshot may be more effective that a single large wound, especially if it is not inflicted in a vital area. It is known that game birds are much more effectively killed when 4-6 smaller size pellets hit them than when only 1-2 large pellets do. Similar logic applies with the buckshot. Additionally, buckshot pellets will almost never give full penetration in the body, thus all energy of the shot is dissipated in the body. Even a soft slug can fully penetrate, especially if the shot did not hit the trunk of the body but an extremity. Another advantage is that buckshot pellets are likely to get easily stopped by walls or other small obstacles and won't travel far and wound bystanders. In contrast, handgun or rifle bullets pose a lot more danger in this case. Shortly, buckshot loads in a shotgun are more safe and efficient at short ranges.
In addition, buckshot gives an advantage in case of hurriedly placed shots - in stressful situations (that law enforcement rightfully anticipates) and in short-range, brush country hunting situations. Buckshot will spread out, just like regular shot, and increase the chance of hitting the target with less than perfect aiming. This is perfectly ok for the law enforcement because even a single pellet hit is likely to distract a culprit enough to allow officers to apprehend him. However, this is where the main drawback of the buckshot lies for the hunting purposes - if a hunter takes a fast, poorly aimed shot, chances are high that the animal will be wounded and never recovered.
Tied buckshot. This is rather rare type of the shotgun load. It must be hand-loaded, but the results might be surprising. Tied buckshot is made by using nylon string to tie together a few large buckshot balls. Typically, this is done using buckshot sizes such that only 9 balls fit in one load, making the loading less laborious. Each ball is slit to approximately half its diameter, and the end of the tying string is pinched by squeezing the ball back together. The procedure is repeated for each ball. The free ends of the strings are simply tied in a knot about 4-6 inches from the balls. It is a very good idea to buffer such loads to prevent the strings from tearing during the shot.
Tied buckshot combines the advantages of slugs and buckshot. Typically, the pattern of such loads is less than 10 inches at any reasonable shotgun distance. The shocking effect on the deer to elk sized game is tremendous at ranges of about 40-70 yards. It must be emphasized that using buckshot of any size is unreasonable at distances exceeding 70 yards in any case.