HOUSTON -- Nestled near the Texas Medical Center and Rice University is one of Houston's affluent neighborhoods. It soon could be the site of a 23-story condominium project.

"I'm going to look at 23 stories," said Earl Martin while standing in his driveway.

Martin lives just a quarter-mile from the proposed Ashby high-rise project.  He's one of hundreds of residents who have been fighting the project for the last two years.

"This is a neighborhood. It's our neighborhood.  A 23-story building next to one- and two-story homes just doesn't belong here," said Martin.

Besides not liking the idea, residents are worried about increased traffic.  Ashby will be built on Bissonnet, a two-lane street.

After recent months of negotiations between Houston's Public Works Department and attorneys for the developer, an agreement has been met.

The new plans reduce the number of residential units in the project from 180 to 120 units. Also eliminated from the plans are a commercial spa, retail space and offices. Construction must start between by 2013 and must be built continuously. Otherwise, a new traffic study would need to be done.

"I thought all along this was an ill-conceived project.  They've made some concessions on the traffic and I think that's good.  A 23-story high-rise does not belong in this neighborhood.  Bissonnet is still not a big enough street to handle it," resident Norm Rund said.

Residents have been vocal about their opposition.  They've posted yard signs and bumper stickers. With the first approval out of 11 set of plans, residents said their feelings haven't changed.

"We're not going to back off.  We are not giving up," said Martin.

KPRC Local 2 called the developers of the Ashby high-rise project and the calls were not returned.