Species of Vanilla were formally classified by Rolfe (1896) into subgeneric sections based primarily on vegetative features and secondarily on flower morphology. Vanilla section Neotropical, leafy, fragrant clade Paleotropical clade V. africana subclade African subclade Caribbean leafless subclade Asian subclade Aphyllae was proposed in order to accommodate all of the leafless species in the genus (e.g. V aphylla, V. barbellata, V. roscheri, and others). All of these are fully photosynthetic plants with green stems, but have either lost their leaves entirely or have modified them into rigid hook-like structures (e.g. V poitaei) to assist with climbing. They are found most commonly in hot, xeric, full sun habitats where leaf reduction is probably an adaptation for water retention. Species within this section can be found in the Caribbean, continental Africa, Madagascar, and Asia. Although some of them produce fleshy fruits, there is no evidence that these are aromatic or worthy of cultivation as crop plants. Rolfe’s classification of these species together implies that they share a recent common ancestor, but molecular studies have demonstrated that this is not the case (Cameron 2005). Instead, there appear to be at least three independent cases of leaf reduction/leaf loss in Vanilla - at least once in Africa, in the Caribbean, and in Asia. The section, therefore, is not monophyletic, but an artificial grouping of species with shared vegetative morphology derived by convergent evolution.
For the remaining species not classified in Vanilla subgenus Aphyllae, Rolfe created section Foliosae. As the name indicates, all of these are leafy. Since this is such a large group, Porteres (1954) later divided the section further into subsections. Vanilla section Foliosae subsection Membranaceae is a small cluster of species characterized by wiry stems, thin leaves, short aerial roots, and flowers in which the labellum is not fused with the column. The labellum also lacks the complex bristles, hairs, and scales characteristic of other Vanilla species, and the fruits tend to dry on the vines and split lengthwise. In some species of this section, flowers are born solitary within leaf axils rather than in congested racemes. Vanilla mexicana exemplifies this subsection, and molecular systematic studies have demonstrated that the group is basal relative to all other species within the genus. Plants tend to grow in wet, shady habitats with rich organic soils, and probably survive in close association with mycorrhizal fungi. They are very difficult to transplant or cultivate, and there is no evidence that the fruits produce aromatic vanillin.
Vanilla planifolia, V. pompona, and many other species of the genus were classified by Porteres into Vanilla section Foliosae subsection Lamellosae. The group is so named because species within this group are characterized by flowers with flattened scale-like appendages (lamellae), and complex ornamentation on their labella. A third subsection, Papillosae, was erected for the 28 or so species characterized by fleshy leaves and flowers usually with thick trichomes running up the center of the labellum, but without lamellate scales. The labellum of all these flowers is always fused to the column along its margins to form a floral tube. Species within Vanilla section Foliosae are pantropical in distribution, but recent molecular systematic studies have demonstrated that this group is also artificial. Instead, species of Vanilla cluster by geography with leafy and leafless species intermixed, as can be seen in Figure 14.2. Specifically, all Old World species (from Africa and Asia) share a common ancestor together with the leafless New World species. The latter were probably dispersed from Africa to the Caribbean via Atlantic hurricanes. All remaining Neotropical species, including V. planifolia, share a different common ancestor. It is within this group that aromatic fruits producing significant levels of vanillin are found. As such, the group has informally been named the “American fragrant species”. Note that molecular studies position V. tahitensis (the only known tetraploid within the genus) inside this group of Neotropical relatives, thereby confirming the hybrid origin of Old World Tahitian vanilla from New World species. Whether the hybrids were man-made by Europeans in the Old World or were naturally occurring in Mesoamerica prior to European contact is uncertain.