No, Azriel is not specifically telling Elain what she can or can't do because he's not her keeper.

What he's saying though, is he doesn't believe she should be exposed to the Trove because he doesn't believe she is capable of handling it's darkness and coming out on top.

Just because he's not telling her “she’s not allowed”, does it make it any better that he doesn't think that she is strong enough to face it?

He also waited until she left the room to voice his concerns. If you are against the idea, then why not admit to her face that you think it's a bad one?