The effective information of 3D multifocal image stack is distributed on different image layers, so the feature extraction and classification of image stacks are significantly different from that of 2D images. In this paper, an image fusion based multilinear analysis approach is presented to use for classification of multifocal image stacks. First, the image fusion techniques are used to combine the relevant information of multifocal images within a given image stack into a single image. Besides, multifocal images within a stack are fused along 3 orthogonal directions, and multiple features extracted from the fused images along different directions are combined by using canonical correlation analysis (CCA). Furthermore, because multifocal image stacks represent the effect of different factorstexture, shape, different instances within the same class and different classes of objects, the image fusion method within a multilinear framework is embeded to propose an image fusion based multilinear classifier. The experimental results demonstrate that the multidirection image fusion based multilinear classifier can reach a higher classification rate (97%) than other classification methods.