AngularJS - Directives vs Controllers
Here's a brief stand-alone answer, with links to official docs for further info (definition of "services" added for good measure):
http://docs.angularjs.org/guide/controller
In AngularJS, a controller is a JavaScript constructor function that is used to augment the AngularJS scope.
When a controller is attached to the DOM via the ng-controller directive, AngularJS will instantiate a new controller object, using the specified controller's constructor function. A new child scope will be available as an injectable parameter to the controller's constructor function as $scope.
http://docs.angularjs.org/guide/directive
At a high level, directives are markers on a DOM element (such as an attribute, element name, or CSS class) that tell AngularJS's HTML compiler ($compile) to attach a specified behavior to that DOM element or even to transform the DOM element and its children.
http://docs.angularjs.org/guide/services
AngularJS services are substitutable objects that are wired together using dependency injection (DI). You can use services to organize and share code across your app.
I think you should only use controllers to wire up services, dependencies & handle business logic.
Directives should be used for DOM manipulation so its great for handling user interactions e.g. add/edit a widget. It would be an anti pattern to add that to a controller as it's not a concern of the controller and it would get bloated by other features, it's easy enough to pass values(scope) from your controller to a directive if required.
You then get the advantage of being able to place your add a widget directive in other places of your app if required, and generally makes the code base easier to follow as its following Law of Demeter
Knowing when and where to split functionality is one of the hardest things for me about Angular you will make mistakes but it gets easier with practice.
To answer the question you probably should split it into a directive(s) but it's also a matter of complexity, If your app is simple like you said you may find what you have will be okay. It may just become a case of refactoring when a requirement changes.
My peers and I ran into a similar situation while learning AngularJS. I've put together this simple table that should give everyone a good starting point depending on what type of functionality they are trying to implement in Angular. http://demisx.github.io/angularjs/2014/09/14/angular-what-goes-where.html