You can’t do this with just ordinary JavaScript; as Levi notes, if you could, that would be a usability and security nightmare.
That said, there are a couple of ways you could get something like this to work. For example, a Firefox add-on can launch external programs, so you could write such an add-on to open your app in IE, and ask your users to install it (or have an admin pre-install it for them, if this is e.g. for an in-house app in a managed workplace setting).
Another possibility might be to use a custom URL scheme (like myapp:) to link to your application, and register that URL scheme to be opened in IE (or, rather, in some kind of a wrapper script that takes the custom URL, converts it to a normal HTTP URL, and opens it in IE). Again, though, this requires configuring (and, probably, installing a script on) the end-user’s computer, so it’s probably only suited for office or similar environments.
Note that both of these solutions have security implications that you should keep in mind. In particular, any add-on or script you use for them should only work for specific URLs that point to your app, and should refuse to open any other URLs someone might try to feed it.
In any case, if you’re trying to do this for a public website, keep in mind that there’s no way you can make this work for Linux / Mac users who don’t have IE to begin with. And even many Windows users might be reluctant to use IE (not to mention installing an untrusted extension!) without a very good reason; after all, there’s presumably a reason why they chose to use Firefox (or Chrome or Opera or whatever) in the first place.