If you wonder why even after calling setAutoCancel(true) after you instantiate Notification.Builder and tapping on the notification it still stays put in the notification area, you did nothing wrong.
You just lacked something for sure. The thing is, you have to set an Intent so that when you tap on the notification, it will execute that Intent. The setAutoCancel(true) will then work and hide the notification.
Here is a sample code that will just hide the notification and does nothing else.
Notification.Builder builder = new Notification.Builder(context)
.setContentIntent(PendingIntent.getActivity(context, 0, new Intent(), 0)).build();