The song was first composed by Robert Cole and J. Rosamond Johnson while they were still performing as the vaudeville act Cole and Johnson. They had intended it for a "comic opera" they called Toloso; the opera was never produced.[3] After a performance, they were discussing the couplet "If you like me, like I like you, No knife can cut our life in two," and Cole suggested they use the idea for a new song.[4] Cole stated he took inspiration from a soldier's story he heard, of a romance he had under a bamboo tree with a Red Cross nurse.[5] At Johnson's insistence, Cole used the melody of the spiritual "Nobody Knows the Trouble I've Seen" as the basis for the song.[6][3]
After writing the words and music, Cole and Johnson sent the song to James Weldon Johnson who made changes to the song. Johnson forwarded the song to a music publisher under the name If You Lak-a-Me, Lak I Lak-a-You. The publisher changed the name to "Under the Bamboo Tree", taken from the final line in the song's chorus.[4]
A month later, Cole and Johnson performed the song at party hosted by George W. Lederer. There, Marie Cahill took a liking to the song, and campaigned against producer Ludwig Englander for its incorporation in the Broadway production of Sally in Our Alley, of which she starred. The song premiered in this musical on August 29, 1902, and it helped make the play successful.[4][7] Cahill included it in Nancy Brown the next season. It went on to become one of two songs that Cahill would commonly interpolate into her own performances per request.[7][3]
Over the course of six months, 400,000 copies of the song were sold.[2] It became a popular and well-known song in the United States, and further spread to England and India.[5][8]