This paper describes a framework for action recognition which aims to recognize the goals and activities of one or more human from a series of observations. We propose an approach for the human action recognition based on the 3D dense micro-block difference. The proposed algorithm is a two-stage procedure: (a) image preprocessing using a 3D Gabor filter and (b) a descriptor calculation using 3D dense micro-block difference with SVM classifier. At the first step, an efficient spatial computational scheme designed for the convolution with a bank of 3D Gabor filters is present. This filter intensifies motion using a convolution for a set of 3D patches and arbitrarily-oriented anisotropic Gaussian. For preprocessed frames, we calculate the local features such as 3D dense micro-block difference (3D DMD), which capture the local structure from the image patches at high scales. This approach is processing the small 3D blocks with different scales from frames which capture the microstructure from it. The proposed image representation is combined with fisher vector method and linear SVM classifier. We evaluate the proposed approach on the UCF50, HMDB51 and UCF101 databases. Experimental results demonstrate the effectiveness of the proposed approach on video with a stochastic textures background with comparisons of the state-of-The-Art methods.