I've run into this several times now when re-deploying servers as iSCSI SAN storage systems for vSphere. What happens is there's an old filesystem partition (or two) on the device\disk so ESXi refuses to configure it as a datastore.
To fix this problem you have to delete these partition(s) from the device\disk.
Word of warning: make sure you delete the correct partition! If you delete the wrong partitions, you may have to recover/re-install ESXi. The correct partitions should not be difficult to find, but now if you screw something up you can't blame me - you've been warned!
Use the vSphere client - on the ESXi host go to the Configuration tab, Storage, Devices. Take note of the device name your trying to configure as a new datastore.
SSH into the ESXi host.
Run the following command:
This will list the partitions on that disk device.
Now you need to delete these partitions:
When prompted, delete each partition. Press "d" for delete, then "1" for partition 1. Do this for all partitions on this device.
When finished, press "w" to write the changes to disk.
You should now be able to go back into the vSphere client and create a new datastore using this device!