Passive endings are -αι in the singular, yes (including verbs with passive endings but active meaning such as έρχομαι, χρειάζομαι, είμαι). And -ε is the ending on the first and second person plural whether active or passive (e.g. ερχόμαστε / θέλουμε - έρχεστε / θέλετε).
But passive third person plural is -αι again, e.g. έρχονται, χρειάζονται, είναι, while if active third person plurals usually end in -ν but this can sometimes be -νε as well depending on dialect -- if that extra vowel does get added on it will be spelled with an epsilon, though!
Two endings to watch out for particularly are -εται + -ετε, which are pronounced identically and can occur on the same verb stem: for example, διαβάζεται "it is read" versus διαβάζετε "you (pl.) read". One is passive, third person singular; the other is active, second person plural.