Digit detection with Nx

95% accuracy is about the limit for a fully connected layer. If you want better accuracy, you will have to have a different architecture; you could consider doing a convnet or if you feel like a real challenge do something like a resnet, which will give you 99.9% accuracy.