The regular season was a success with the Browns finishing with a regular season record of 10–3–1.They were coached by Blanton Collier who had replaced the historic Paul Brown the previous season. The team had a tremendous amount of heart, which was demonstrated by the fact that they had key commanding wins throughout the season. For instance, they swept their arch rival New York Giants, who the previous year had edged them out as the eastern conference champion. Not only did they win both times that they played against the Giants but both wins were very convincing, the first being a 42–20 home victory and the second being a 52–20 away victory. The second victory over the Giants was a clutch, season ending game that clinched the eastern conference title. Many of the Browns' wins during the regular season were in a very commanding manner, with a 37–21 win over the Detroit Lions being a prime example. The win over the Lions carried extra significance due to the fact that the Lions had been the team that knocked them out of the conference champion hunt the previous season by beating them 38–10 in the second to last regular season game.
The Browns were led by Hall of Fame running back Jim Brown who had a stellar regular season, rushing for 1,446 yards with a 5.2 yards/carry average. Although they had a great rushing game, the Browns had a very balanced offense, choosing not to just hand the ball to Brown on every play. The quarterback of the team was Frank Ryan who had a decent season throwing for 2,404 yards and 25 touchdowns while throwing 19 interceptions. The top receivers of the team were Paul Warfield and Gary Collins, the second of whom would become a legend by catching three touchdowns in the championship game against the heavily favored Baltimore Colts.
Leading into the game, the Browns were huge underdogs. Most experts had them losing by double digits. Baltimore was so heavily favored that after the Browns won the game, Sports Illustrated had to scramble to find a picture of a Browns player to put on its cover. Baltimore had the league's best offense and had a league best record of 12–2. They were stacked with future Hall of Famers such as Johnny Unitas, Lenny Moore, and John Mackey. The Browns though, were unfazed by the apparent talent disparity and Jim Brown was reported stating before the game, "we're going to kick their [butt] today." The game-time temperature that day was 34 degrees and felt much colder in 15- to 25-mph winds whipping under gray December sky. The Municipal Stadium crowd of 79,544 was the second largest in NFL title-game history at the time. The Browns knew that if they wanted to be in the game they had to make a statement early on, and they did just that. Galen Fiss, the Browns team captain, broke up a screen pass from Unitas to Moore, sending Moore airborne for a loss. The Browns tenacity on defense is what got them to the half time score of 0–0. Brown's running back Ernie Green reported after the game about half time, "We cleaned ourselves and sat down, and it seemed like something came over all of us. I think we all kind of looked at each other and concluded, 'Hey, we can beat these guys.'" Not only did the Browns "beat" the Colts in the second half, They destroyed them, scoring 27 unanswered points. Gary Collins became a Cleveland Browns legend by catching three touch down passes, the third one being a 51-yarder with Colts defender Bobby Boyd all over him. The biggest story of the game was how well Cleveland's defense played against Baltimore's heralded offense. Cleveland was able to hold Unitas to just 95 yards while intercepting him twice.
This was the last major sports championship won by a Cleveland-based team until 2016, when the Cleveland Cavaliers, a team that was formed in 1970, defeated the defending champion Golden State Warriors in a seven-game NBA Finals. Not only is it remembered in Cleveland but ESPN ranks the '64 title game as the second-greatest NFL postseason upset, behind only Joe Namath's guaranteed win over the Colts in Super Bowl III four seasons later.