In his childhood, Ballard moved around cities such as Louisville, Kentucky and areas in Indiana. He said that he moved around two to three times each year. Ballard did not know his own father while growing up, and his mother only introduced him when he was nine years old. He told one newspaper, the Dayton Daily News that two of his other brothers, Corey and BJ, went to jail in 2010 after running away from the police with possession of a gun. Ballard's youngest brother, Ricky, had experience in jail as well. In addition, some of his cousins had been shot or killed. Antonio himself had a gun pointed in his face.[3]
The presence of gangs limited the time Ballard could practice basketball. Since many of them were his family members, he was allowed to shoot. He said, "It was rough basketball—kind of like prison ball, I guess. Fights would break out and you had to hold your own, but you couldn't take it too far or something bad would happen."[3] In Jeffersonville, Indiana, there was an old basketball rim, where gangs such as the "Money Hungry Gangsters" would park their cars. Ballard said, "I've taken the good parts of my past life—and one thing you do get from a gang is having each other's back."
Antonio Ballard usually hits his chest with 4 fingers when making a shot which is meant for a special someone. [3]