James Yancey was the oldest of four children including a sister (Martha) and two brothers (Earl and John), who later began doing music as Illa J. The family lived at a corner house near McDougall and Nevada, on the east side of Detroit.[4] He developed a vast musical knowledge from his parents (his mother is a former opera singer and his father was a jazz bassist).