It's not a hard and fast rule, but basically any word ending in a consonant is going to be Masculine, words ending in -a or -я are going to be feminine (very few, such as бремя, племя, пламя, знамя, имя, for example, are neuter), words ending in -е or -о are neuter. Words ending in soft sign just require you to memorize them, since some are feminine and some are masculine.
@melinski_s - Words that end in a consonant or -й are masculine, words that end in -а/-я are generally Feminine (there are about seven or eight neuter words that end in -я, though, like время, бремя, пламя, знамя, имя), and words that end in -o/-e are generally neuter (except for кофе - coffee, which is masculine, though you will see some Russians use it like neuter). Also foreign words like такси are generally neuter.
For words that end in a soft sign, you're going to have to just learn because some are masculine and some are feminine.