I'm new to these, but I believe the answer is yes.
Assuming your external interface IP is 4.4.4.4 and you want to restrict access to only 8.8.8.8, I think this would work.
conf t
firewall Internet
policy 1001 in address 8.8.8.8 32 4.4.4.4 32 service ssh self
exit policy
or to restrict by network 8.8.8.0/24 >
conf t
firewall Internet
policy 1001 in address 8.8.8.0 24 4.4.4.4 32 service ssh self
exit policy
or you could replace ssh with telnet if you like living on the edge.
*this also assumes that you have already enabled SSH and/or Telnet