A dynamic fuzzy neural network algorithm (D-FNN) is proposed. In practical application, Kalman filter (KF) is used to adjust the result parameters of D-FNN. Meanwhile, extended Kalman filter (EKF) is used to update the center and width of the premise parameters, so that all parameters can be modified. This algorithm is used to smooth, filter or predict the state of a nonlinear dynamic system. Comparing to other online algorithms which are based on gradient, EKF can accelerate the convergence speed of D-FNN. The total least squares (TLS) is a pruning technique to select the important fuzzy rules of D-FNN, which also is a technique to compensate data error in linear parameter estimation problem. If the inactive fuzzy rules are detected and removed during the learning process, a more compact D-FNN structure can be obtained. Finally, a simulation analysis for actual case verify the effectiveness and efficiency of the algorithm.