Jérémy started his career with one of the biggest clubs in the Doubs department, FC Sochaux-Montbéliard. He made his Ligue 1 debut, during the 2002/2003 season, in a league match against CS Sedan coming on as a late-match substitute. Despite being 18, he was inserted into the starting 11, and quickly provided an impact scoring his first professional goal in just his third league match in a 2–2 draw against Rennes. He made 23 league appearances that season scoring 4 goals, which included, excluding his first, an equalising goal against Paris Saint-Germain, and two game-winning goals against Nantes and Le Havre. His play, along with the team itself, was a main factor in the club finishing 5th and qualifying for the UEFA Cup.
Over the next two seasons, Jérémy continued his great form appearing in 63 league matches and scoring 6 league goals. He also performed well in Europe appearing in 14 matches and scoring 2 goals. He also helped Sochaux win the Coupe de la Ligue in 2004. His performances at Sochaux led to strong interest from Italian club Juventus, English clubs Newcastle United, Everton, and Southampton.
Toulouse FC
After announcing his desire to move to another club and only having one year left on his current deal, Sochaux agreed to listen to offers for the player. Eventually, he agreed to join Toulouse after agreeing to a four-year deal. He made his league debut for Toulouse against his former club Sochaux playing the full 90 minutes in a 1–0 victory. He continued this positive play appearing in 36 league matches and scoring two goals against AC Ajaccio and Troyes. The following season, he made 32 league appearances and score another two goals helping Toulouse qualify for the UEFA Champions League. However, both Mathieu and Toulouse had horrible 2007/2008 seasons with Mathieu fracturing a bone in his foot, which forced him to miss half the season, and Toulouse finishing just one spot short of relegation. This led to Mathieu pondering his future with the Midi-Pyrénées-based side.
Despite heavy interest from Italian side Roma and other Italian outfits, Toulouse refused to budge with efforts to get Mathieu to remain at the club. Mathieu responded by turning down a contract extension, which effectively allowed him to leave on a Bosman transfer this upcoming summer. Eventually it was agreed by Toulouse and Mathieu that the player will see out the season at Toulouse, after Toulouse turned down a ¤4.5 million move to Bordeaux.
Despite the ongoing transfer situation, Jérémy has remained civil in regards to playing. He has made 27 league appearances so far this season, contributing to Toulouse's successful league campaign, as well as their success in the Coupe de France, where they were eliminated in the semi-finals by the eventual champions En Avant Guingamp.
Valencia CF
On 10 June 2009, it was announced that Jérémy would be Unai Emery's first signing of the summer for Spanish giants Valencia after securing the left back on a three-year deal. The transfer came into effect on 1 July 2009.
Mathieu had a slow start to his Valencia career, making a couple of howlers in his first few games, but his play progressed nicely throughout the season, and he is now considered to be arguably VCF's best defender.
