Using Associativy's access control services you can leverage Orchard user roles to control who can view graphs.
Enable the Associativy Frontend Authorization Administration feature of the Associtaivy.Administration module. This will instantly use the most restrictive policy: nobody will be able to view any graphs.
To allow users of certain roles to be able to view graphs, go to each graph's admin page and setup access control under "Users authorized to view the graph".
Note that the default admin user has no roles; to use frontend engine authorization administration, add a role to it.