Sorting posts by a number

Hi!

I want to sort my posts by a custom field that contains only a number, like “354”.

Tried the sort_by_title plugin but couldn’t get i to sort properly by only the numbers. The output would be something like:

899
560
124
230
324

and so on. I wan’t them to sort by the lowest number first. If i sort by letters i works fine, but not with numbers. Any pointers on how to make it happend?

Thanks!