If preventing an increase in the consumption of state-provided benefits justifies restricting freedom in immigration, then it also justifies restricting freedom in, well, anything.
Borders are an issue that can be used to determine what type of “libertarian” you are dealing with. Borders in and of themselves necessitate coercive violence. They are indefensible even if we take great liberty with what aggression means in the context of the nonaggression principle. If individuals have rights, said rights are natural or human rights and all individuals have equal rights then they either do or do not have a right to be free from aggression.
Many “libertarians” profess to adhere to the NAP as an ethical code or deontological rule. Yet in the context of immigration they switch to a discussion on the *consequences* of immigration. Well, peep this:
If individuals have a right to move freely without being aggressed upon then they have that right regardless of what economic or political consequences arise. Even if we assumed immigration would be a disaster, even if we assume a worst case scenario, it would not matter. Even if it meant that the United States or any other state would fail, it still would not matter. This is an indictment not of the immigrant, but of the state they immigrate into. This tells us that a state or its economy is not truly viable in the absence of coercive force used to restrict movement.
In fact, this is also an indictment of anyone who claims to be “free market” and supports closed borders. It lets you see exactly up to what limit that market will be “free.” You can measure that freedom in latitude and longitude.