In Demarest, the average residential property-tax bill rose by $1,371 in one year, landing at $26,112 in 2025. At the same time, the borough’s latest Census estimate remains below its 2020 estimates base.
That strange split is not confined to one expensive Bergen County ZIP code. It shows up from Sussex County’s rural roads to Hudson County’s apartment blocks and the Shore.
Every place on this list had a lower 2025 estimated population than its 2020 estimates base, while its average residential tax bill increased from 2024 to 2025. That does not mean taxes alone sent anyone packing; people move for jobs, housing, family, retirement, and plenty of other reasons.
Still, in a state where carrying costs can dominate a home-buying conversation, the pattern is hard to ignore. Some declines are modest and others are sharper, but all ten towns show the same uncomfortable contrast: the resident count is drifting down while the annual bill keeps moving up.
1. Highlands
Start at the water, because that is what makes Highlands such a striking entry here. This compact Monmouth County borough sits on Sandy Hook Bay, pairs Shore-town character with high-speed ferry access to Manhattan, and offers the kind of location that should give it plenty of staying power.
Yet the latest Census estimates put Highlands at 4,350 residents in 2025, down from a 2020 estimates base of 4,625. That is a drop of roughly 5.9%, the steepest percentage decline among the ten places on this list.
The tax trend is headed the other way. New Jersey’s average residential property-tax bill for Highlands climbed from $8,443 in 2024 to $8,985 in 2025, an increase of about 6.4%.
In absolute terms, that bill is still lower than the eye-watering averages in several affluent Monmouth County communities, but the one-year jump matters. Highlands has always offered a distinctive combination: waterfront views, access to the Shore, and a realistic commute to New York without giving up a small-town footprint.
That makes the population slide more notable, not less. A tax bill does not explain every move, and no one should read these figures as a mass exodus.
But when a town loses nearly 6% of its estimated population while the average residential tax bill adds more than $500 in a year, the affordability conversation gets harder to wave away.
2. Andover Township
Drive northwest into Sussex County and the scenery changes completely. Andover Township is rural by North Jersey standards, spread across wooded roads and open land about 40 miles northwest of New York City, with the quieter pace that draws people who would rather see trees than a wall of high-rises outside the window.
Its numbers, though, point to a familiar squeeze. The 2025 Census estimate is 5,721 residents, down from a 2020 estimates base of 6,019, a decline of about 5%.
Meanwhile, the average residential property-tax bill rose from $10,687 in 2024 to $11,078 in 2025. That 3.7% increase is not the biggest jump on this list, but it lands differently in a community where homeownership is the norm: Census data put the owner-occupied housing rate above 90%.
In other words, property taxes are not an abstract line item affecting a narrow slice of households here. For many residents, they arrive directly with the cost of owning the house.
Andover still has the appeal that makes Sussex County attractive to people seeking space, privacy, and a less hectic daily rhythm. The catch is that rural does not automatically mean inexpensive.
When taxes clear $11,000 on average and the population is moving the opposite direction, Andover becomes a useful example of how New Jersey’s cost pressure extends well beyond the commuter suburbs closest to Manhattan.
3. Chesterfield Township
This Burlington County township makes the list with an especially interesting twist: it was still a growth story going into the beginning of this decade.
The 2020 Census counted 9,422 residents, while Chesterfield retains an older identity through the Crosswicks and Chesterfield historic districts and roots stretching back to early Quaker settlement.
More recently, the direction has softened. The 2025 estimate is 9,046 residents, compared with a 2020 estimates base of 9,376, a decline of roughly 3.5%.
At the same time, the average residential property-tax bill moved from $12,692 in 2024 to $13,321 in 2025, a one-year increase of about 5%. Chesterfield is also overwhelmingly owner-occupied, which makes the tax story particularly immediate for households rather than something experienced mostly through rent.
The township is not a dense rail suburb or a Shore community with a huge seasonal pulse; its character mixes newer residential development, preserved farmland, and old village streets. The recent population dip does not erase the growth that came before it, and it would be a stretch to pin the change on taxes alone.
Still, the combination is worth watching. Adding residents is one challenge. Keeping households for the long haul is another once mortgages, assessments, and an annual property-tax bill above $13,000 all enter the equation.
4. North Bergen Township
Few places on this list feel less like the stereotype of a “town losing people” than North Bergen.
The Hudson County township is packed tightly along the Palisades, with apartment buildings, busy commercial corridors, and James J. Braddock North Hudson County Park giving residents a sizeable green break from the surrounding urban grid.
Yet Census estimates put the 2025 population at 61,483, down from a 2020 estimates base of 63,374, a decline of about 3%.
Its average residential property-tax bill rose from $8,505 in 2024 to $8,936 in 2025, roughly a 5.1% increase. The important wrinkle here is ownership.
Only about 42% of occupied housing units are owner-occupied, far below the rates in places such as Rumson or Fair Haven. That means a headline about property taxes does not translate into every household opening the same kind of bill.
For renters, property costs are more likely to be felt indirectly through the broader economics of housing. North Bergen also remains deeply international, with a large foreign-born population and households where languages other than English are common.
The decline is still substantial—nearly 1,900 residents compared with the 2020 base—but North Bergen is a useful reminder that tax pressure works differently in a dense, renter-heavy community than it does in a suburb dominated by single-family homeowners.
5. Fair Haven
An $18,950 average property-tax bill gets your attention even before you look at the population chart. In Fair Haven, that was the 2025 average residential bill, up from $17,848 a year earlier—a jump of $1,102, or about 6.2%.
The borough’s estimated population, meanwhile, slipped from a 2020 base of 6,269 to 6,096 in 2025, a decline of roughly 2.8%. Fair Haven has been closely linked to the Navesink River throughout its history, and that connection remains easy to see at places like the municipal dock and riverfront parks.
Its small scale and largely residential character help explain why it remains such a coveted corner of Monmouth County. Nearly 90% of occupied homes are owner-occupied, so property taxes are a direct household expense for most residents rather than a distant concern.
This is also why Fair Haven complicates the easy assumption that rising tax burdens only become meaningful in less affluent towns. An annual increase north of $1,000 is real money even where home values and incomes run high.
The population change is not dramatic enough to call an exodus, and taxes are only one possible piece of the picture.
Still, the numbers create an unmistakable tension: Fair Haven remains the kind of river-adjacent borough people work hard to buy into, yet the latest estimates show slightly fewer people calling it home as the average cost of keeping that home climbs.
6. Rumson
A river town filled with expensive homes might seem like an odd place to look for an affordability story, which is exactly what makes Rumson useful here. Bounded by the Navesink and Shrewsbury rivers, the borough has long been associated with large properties, leafy residential streets, and some formidable home prices.
Homeownership is exceptionally common, with roughly 93% of occupied housing units owner-occupied. The average residential property-tax bill is just as formidable: $23,607 in 2025, up from $22,890 in 2024.
That is an increase of $717 in a single year. At the same time, the population estimate fell from a 2020 base of 7,326 to 7,157 in 2025, a decline of about 2.3%.
Rumson still has the ingredients that make a small river community sticky, from waterfront access to neighborhood gathering places such as Victory Park, whose courts and playground received recent renovations. But wealth does not make carrying costs invisible.
A household that can afford an expensive home may have more room in the budget, yet a property-tax bill above $23,000 still becomes part of the calculation when families consider staying, downsizing, or relocating. None of that proves taxes caused Rumson’s population decline.
What the figures do show is that even one of New Jersey’s priciest residential markets is not immune to the same basic pattern: fewer estimated residents and a higher annual tax burden.
7. Madison
The railroad is woven so deeply into Madison’s identity that the town center practically grew around it. The line arrived in the 19th century, helped fuel the local rose-growing industry that produced the enduring “Rose City” nickname, and today still gives commuters direct NJ Transit service toward New York.
Add a compact downtown and nearby college campuses, and there are plenty of reasons for residents to stay put. Even so, the latest Census estimate is 16,607, down from a 2020 estimates base of 16,963, a decline of about 2.1%.
Property taxes moved briskly in the other direction. The average residential bill rose from $14,959 in 2024 to $15,897 in 2025, an increase of $938, or roughly 6.3%.
That is a noticeable one-year jump in a housing market where the cost of buying was hardly modest to begin with. Madison’s population decline is relatively small, and it would be careless to treat one expense as the explanation for every person who leaves.
But that is what makes the borough revealing. It has the train, the downtown, the history, and the institutional anchors that usually help a suburb hold its appeal.
When even that package comes with a rapidly rising average tax bill and a slightly shrinking resident count, the affordability problem becomes harder to dismiss as something that only affects towns without obvious advantages.
8. Margate City
Summer can fool the eye in Margate. Beach days, second homes, packed seasonal events, and the unmistakable sight of Lucy the Elephant give this Atlantic County city a warm-weather pulse much bigger than its year-round resident count.
Margate even hosts Beachstock, billed by the city as the Shore’s largest beach party, while Lucy has been part of the local landscape since 1881. That seasonal distinction matters when reading the numbers.
The 2025 Census estimate puts Margate at 5,194 residents, down from a 2020 estimates base of 5,313, a decline of about 2.2%. Over the same recent stretch, the average residential property-tax bill rose from $9,917 in 2024 to $10,545 in 2025, an increase of about 6.3%.
The housing market provides even more context: state records show the average price among recorded residential sales in 2025 was about $1.32 million. For owners deciding whether a Shore property remains a primary residence, a second home, or something they eventually sell, annual carrying costs matter.
Margate’s figures therefore deserve more nuance than “people are fleeing.” A busy July weekend tells you almost nothing about how many people live there in February.
The city remains a major Shore draw; what has slipped is its estimated full-time population, while the average tax bill attached to residential property has moved sharply upward.
9. Demarest
Here is the number that makes Demarest jump off the page: $26,112. That was the Bergen County borough’s average residential property-tax bill in 2025, the highest among the ten communities on this list.
A year earlier, the average was $24,741, so the increase came to $1,371—about 5.5%—in just one year. The population change is smaller but points the other way.
Census estimates put Demarest at 4,893 residents in 2025, compared with a 2020 estimates base of 4,980, a decline of 87 people, or roughly 1.7%. In a borough this small, that deserves perspective: a few dozen households can noticeably move the percentage, so “exodus” would be doing too much work.
Still, the tax figure is difficult to shrug off. Demarest is an established Northern Valley community with deep Bergen County roots and reminders of its past that include its historic railroad station.
It is also an extremely expensive housing market; state records put the average price of recorded residential sales in 2025 at roughly $1.92 million. That makes the cost of entry formidable, and the cost of staying is not exactly pocket change either.
Demarest’s inclusion is less about dramatic population loss than about direction: a modest resident decline paired with this list’s biggest average tax bill and one of its largest one-year dollar increases.
10. Union City
Finish with the place where the population loss is biggest in raw numbers—and where the property-tax story needs the most context. Union City’s 2025 Census estimate is 66,656, down from a 2020 estimates base of 68,641.
That is a decline of 1,985 residents, the largest absolute drop among these ten communities. The average residential property-tax bill also rose, from $9,138 in 2024 to $9,383 in 2025, an increase of about 2.7%.
But Union City is not a homeowner-dominated suburb. Only about 19% of occupied housing units are owner-occupied, so most residents are renters rather than people receiving property-tax bills directly.
More than half of residents are foreign-born as well. That makes it especially important not to turn correlation into a tidy cause-and-effect story.
Housing costs can reach renters through many channels, and population change in a dense, mobile city can reflect household size, migration, jobs, rents, and housing availability as much as any single tax figure. What has not disappeared is Union City’s identity.
Bergenline Avenue remains a commercial and cultural spine, with Central American, South American, and Caribbean influences visible in its shops and restaurants. The result is a place that can feel intensely full even as official estimates decline.
Union City closes the list with the clearest lesson: rising property taxes are part of New Jersey’s affordability puzzle, but they are not the whole puzzle.