Tommy Angelo was an ordinary taxi driver trying to make a living on the streets of Lost Heaven until one day an unexpected meeting changed his life forever. Two men jump into his cab, telling him to drive as fast as he can. Bullets begin to hit the cab, and a car with armed pursuers gets closer. Barely escaping a violent death, Tommy obeys the instructions of the two men and delivers them to a bar of their choice.

Impressed with his driving skills, the men pay him and offer him a job in the mafia. The next day, while Tommy is taking a coffee break, his cab is smashed by two rival gangsters. After this, he recalls the proposition from yesterday, and, without much hesitation, joins the Salieri family, making his first steps on the path of organized crime. While performing missions for Salieri, Tommy gradually begins to regret his choice. However, it turns out that joining the mafia was much easier than leaving it.

Mafia is an action and driving game set in the 1930s in Lost Heaven, a fictional city in the USA modeled after New York and Chicago of the Prohibition Era. Similar to GTA games, it consists of free-roaming (driving or on foot) in a large city, completing missions to advance the narrative. The missions often include driving to various locations, car chases, and one race; however, most of them are structured like fairly large and long third-person shooter levels.

The player will have the chance to drive over sixty vehicles that are reminiscent of the 1930's period. Each car handles it differently, with various degrees of damage. It is possible to smash windows, destroy bumpers, lights, and mirrors, dent the car, shoot out the tires, or shoot the tires so much that they fall off. To add to the realism, if the car's gas tank gets punctured, gas will slowly leak out until the car totally empties. There are gas stations scattered throughout the city, allowing the player to refuel. With these vehicles, the player is able to explore twelve square miles of the city, visiting areas such as Central Island, Chinatown, and the Downtown district.

Complementing the vehicles is an array of weapons, such as pistols, tommy guns, shotguns, explosives, baseball bats, and crowbars. While driving a car, Tommy can perform a drive-by, sticking his hand out the window and firing. Trying to impede him is the police force; they will act on anything suspicious. If they see Tommy carrying a weapon, they will attempt to arrest him. Going over the speed limit, running red lights, crashing into buildings, cars, or objects will result in fines.

Tommy is often given new cars to use during the missions, usually provided by Ralph, the mechanic of the Salieri family. The player can also save hijacked cars in the backyard of the bar belonging to Salieri. Weapons can be acquired at the beginning of a mission by Vincenzo, the local arms dealer.