All that daily volume of microsecond stock trades lowers the "friction" of trading considerably: It decreases the difference between buy and sell prices. Get rid of it and that difference affects everybody in the market.
I also think your worry over poor people not paying is overblown. If they did pay a lower limit, that limit would be either inconsequential or significant. If inconsequential, why bother collecting it at some expense that their payment would probably not cover. If significant, then you are taking an important part of their income! $50 may not mean much to you, but it could be a week's worth of food for a poor family. Income disparity is so high in this country that you will never come up with an acceptable minimum tax payment.
I could be one of those "rich" people you are complaining about who doesn't pay. I could make $150,000 in long term capital gains and pay no tax on it. Because in prior years I lost money in long term capital investments and I have $150,000 in loss-carry-forward. Complex? Yes Unfair? No
The other big complexity is depreciation. I buy farm equipment with a limited useful life. So I have to track the loss of that equipment's value every year. That is deductible against the income I make from the farm. Complex? Yes Unfair? No