Hi JZ,
You are correct. You do not need to assign Exchange Online license for resource mailbox. To troubleshoot this issue, please refer to the following steps to create a room mailbox via PowerShell.
1. Right click Windows PowerShell and click Run as administrator.
2. Then click Yes in the following pop-up window.
3. When the Windows PowerShell window open, run the commands in Step 4 and Step 5 in the article below:
http://help.outlook.com/en-us/140/Cc952756.aspx#Step4
http://help.outlook.com/en-us/140/Cc952756.aspx#Step5
4. Then run the commands in Step 2, 4, 5 under “Connect Windows PowerShell on your local computer to the cloud-based service” in the article below:
http://help.outlook.com/en-us/140/Cc952755.aspx
After connecting it successfully, you will receive the following page.
5. Then run the following command to create a room mailbox.
New-Mailbox -Name "Test Room Mailbox" -Aliasrm2–Room
You should replace “Test Room Mailbox” with the mailbox name you prefer to use. Replace “rm2” with the alias you prefer to use.
After that, please check if the mailbox shows up in Exchange Control Panel. You can configure other options for this mailbox.
Thanks,
Monica Tong