Visualization and Intelligent Systems Laboratory
VISLab

 

 

Contact Information

VISLab
Winston Chung Hall Room 216
University of California, Riverside
900 University Avenue
Riverside, CA 92521-0425


Tel: (951)-827-3954

CRIS
Bourns College of Engineering
UCR
NSF IGERT on Video Bioinformatics

UCR Collaborators:
CSE
ECE
ME
STAT
PSYC
ENTM
BIOL
BPSC
ECON
MATH
BIOENG
MGNT

Other Collaborators:
Keio University

Other Activities:
IEEE Biometrics Workshop 2014
IEEE Biometrics Workshop 2013
Worshop on DVSN 2009
Multibiometrics Book

Webmaster Contact Information:
Michael Caputo
michael.vislab@gmail.com

Last updated: June 15, 2016

 

 

Learning, Computer Vision, and Pattern Recognition

Image Super-resolution by Extreme Learning Machine

Image super-resolution is the process to generate high resolution images from low-resolution inputs. An efficient image super-resolution approach based on the recent development of extreme learning machine (ELM) is proposed in our research. We aim at reconstructing the high-frequency components containing details and fine structures that are missing from the low-resolution images. In the training step, high-frequency components from the original high-resolution images as the target values and image features from low resolution images are fed to ELM to learn a model. Given a low-resolution image, the high-frequency components are generated via the learned model and added to the initially interpolated low-resolution image. Experiments show that with simple image features our algorithm performs better in terms of accuracy and efficiency with different magnification factors compared to the state-of-the-art methods.

Real-Time Pedestrian Tracking with Bacterial Foraging Optimization

We introduce swarm intelligence algorithms for pedestrian tracking. In particular, we present a modified Bacterial Foraging Optimization (BFO) algorithm and show that it outperforms PSO in a number of important metrics for pedestrian tracking. In our experiments, we show that BFO’s search strategy is inherently more efficient than PSO under a range of variables with regard to the number of fitness evaluations which need to be performed when tracking. We also compare the proposed BFO approach with other commonly-used trackers and present experimental results on the CAVIAR dataset as well as on the difficult PETS2010 S2.L3 crowd video.

Zombie Survival Optimization: A Swarm Intelligence Algorithm Inspired By Zombie Foraging

Search optimization algorithms have the challenge of balancing between exploration of the search space (e.g., map locations, image pixels) and exploitation of learned information (e.g., prior knowledge, regions of high fitness). To address this challenge, we present a very basic framework which we call Zombie Survival Optimization (ZSO), a novel swarm intelligence ap- proach modeled after the foraging behavior of zombies. Zombies (exploration agents) search in a space where the underlying fitness is modeled as a hypothetical air- borne antidote which cures a zombie’s aliments and turns them back into humans (who attempt to survive by exploiting the search space). Such an optimization al- gorithm is useful for search, such as searching an image for a pedestrian. Experiments on the CAVIAR dataset suggest improved efficiency over Particle Swarm Op- timization (PSO) and Bacterial Foraging Optimization (BFO). A C++ implementation is available.

Dynamic Bayesian Networks for Vehicle Classification in Video

Shadow removal and obtaining the bounding box. Vehicle classification has evolved into a significant subject of study due to its importance in autonomous navigation, traffic analysis, surveillance and security systems, and transportation management. We present a system which classifies a vehicle (given its direct rear-side view) into one of four classes Sedan, Pickup truck, SUV/Minivan, and unknown. A feature set of tail light and vehicle dimensions is extracted which feeds a feature selection algorithm. A feature vector is then processed by a Hybrid Dynamic Bayesian Network (HDBN) to classify each vehicle.

Tracking Pedestrians with Bacterial Foraging Optimization Swarms

Pedestrian tracking is an important problem with many practical applications in fields such as security, animation, and human computer interaction (HCI). We introduce a previously-unexplored swarm intelligence approach to multi-object monocular tracking by using Bacterial Foraging Optimization (BFO) swarms to drive a novel part-based pedestrian appearance tracker. We show that tracking a pedestrian by segmenting the body into parts outperforms popular blobbased methods and that using BFO can improve performance over traditional Particle Swarm Optimization and Particle Filter methods.

Unsupervised Learning for Incremental 3-D Modeling

Learning based incremental 3D modeling of traffic vehicles from uncalibrated video data stream has enormous application potential in traffic monitoring and intelligent transportation systems. In our research video data from a traffic surveillance camera is used to incrementally develop the 3D model of vehicles using a clustering based unsupervised learning. Geometrical relations based on 3D generic vehicle model map 2D features to 3D. The 3D features are then adaptively clustered over the frames to incrementally generate the 3D model of the vehicle. Results are shown for both simulated and real traffic video. They are evaluated by a structural performance measure.

Tracking Multiple Objects in Non-Stationary Video

One of the key problems in computer vision and pattern recognition is tracking. Multiple objects, occlusion, and tracking moving objects using a moving camera are some of the challenges that one may face in developing an ef- fective approach for tracking. While there are numerous algorithms and approaches to the tracking problem with their own shortcomings, a less-studied approach considers swarm intelligence. Swarm intelligence algorithms are often suited for optimization problems, but require advancements for tracking objects in video. We present an improved algorithm based on Bacterial Foraging Optimization in order to track multiple objects in real-time video exposed to full and partial occlusion, using video from both ¯xed and moving cameras. A comparison with various algorithms is provided.

Multi-object Tracking in Non-Stationary Video using Bacterial Foraging Swarms

One of the key problems in the field of image processing is object tracking in video. Multiple objects, occlusion, and non-stationary video are some of the challenges that one may face in developing an effective approach. A less-studied approach considers swarm intelligence. We present a new and improved algorithm based on Bacterial Foraging Optimization in order to track multiple objects in real-time video exposed to full and partial occlusion, using video from both fixed and moving cameras. A comparison with various algorithms is provided.

Feature Fusion of Side Face and Gait for Video-Based Human Identification

Video-based human recognition at a distance remains a challenging problem for the fusion of multimodal biometrics. We present a new approach that utilizes and integrates information from side face and gait at the feature level. The features of face and gait are obtained separately using principal component analysis (PCA) from enhanced side face image (ESFI) and gait energy image (GEI), respectively. Multiple discriminant analysis (MDA) is employed on the concatenated features of face and gait to obtain discriminating synthetic features. The experimental results demonstrate that the synthetic features, encoding both side face and gait information, carry more discriminating power than the individual biometrics features, and the proposed feature level fusion scheme outperforms the match score level and another feature level fusion scheme.

Multiclass Object Recognition Based on Texture Linear Genetic Programming

We present a linear genetic programming approach, that solves simultaneously the region selection and feature extraction tasks, that are applicable to common image recognition problems. The method searches for optimal regions of interest, using texture information as its feature space and classification accuracy as the fitness function. Texture is analyzed based on the gray level cooccurrence matrix and classification is carried out with a SVM committee. Results show effective performance compared with previous results using a standard image database.

Visual Learning by Evolutionary and Coevolutionary Feature Synthesis

We present a novel method for learning complex concepts/hypotheses directly from raw training data. The task addressed here concerns data-driven synthesis of recognition procedures for real-world object recognition. The method uses linear genetic programming to encode potential solutions expressed in terms of elementary operations, and handles the complexity of the learning task by applying cooperative coevolution to decompose the problem automatically at the genotype level. Extensive experimental results show that the approach attains competitive performance for three-dimensional object recognition in real synthetic aperture radar imagery.

Evolutionary Feature Synthesis for Facial Expression Recognition

We propose a novel genetically inspired learning method for facial expression recognition (FER). Our learning method can select visually meaningful features automatically in a genetic programming-based approach that uses Gabor wavelet representation for primitive features and linear/nonlinear operators to synthesize new features. To make use of random nature of a genetic program, we design a multi-agent scheme to boost the performance. We compare the performance of our approach with several approaches in the literature and show that our approach can perform the task of facial expression recognition effectively.

Evolutionary Feature Synthesis for Object Recognition

We propose a coevolutionary genetic programming (CGP) approach to learn composite features for object recognition. The motivation for using CGP is to overcome the limitations of human experts who consider only a small number of conventional combinations of primitive features during synthesis. CGP, on the other hand, can try a very large number of unconventional combinations and these unconventional combinations yield exceptionally good results in some cases. The comparison with other classical classification algorithms is favourable to the CGP-based approach proposed in our research.

Object Detection via Feature Synthesis Using MDL-Based Genetic Programming

We use genetic programming (GP) to synthesize composite operators and composite features from combinations of primitive operations and primitive features for object detection. The motivation for using GP is to overcome the human experts' limitations of focusing only on conventional combinations of primitive image processing operations in the feature synthesis. GP attempts many unconventional combinations that in some cases yield exceptionally good results. Our experiments, which are performed on selected training regions of a training image to reduce the training time, show that compared to normal GP, our GP algorithm finds effective composite operators more quickly and the learned composite operators can be applied to the whole training image and other similar testing images.

Active Concept Learning in Image Databases

We present an active concept learning approach based on the mixture model to deal with the two basic aspects of a database system: the changing (image insertion or removal) nature of a database and user queries. To achieve concept learning, we a) propose a new user directed semi-supervised expectation-maximization algorithm for mixture parameter estimation, and b) develop a novel model selection method based on Bayesian analysis that evaluates the consistency of hypothesized models with the available information. Experimental results on Corel database show the efficacy of our active concept learning approach and the improvement in retrieval performance by concept transduction.

MDL-based Genetic Programming for Object Detection

Genetic programming (GP) is applied to synthesize composite operators from primitive operators and primitive features for object detection. To improve the efficiency of GP, smart crossover, smart mutation and a public library are proposed to identify and keep the effective components of composite operators. To prevent code bloat and avoid severe restriction on the GP search, a MDL-based fitness function is designed to incorporate the size of composite operator into the fitness evaluation process. The experiments with real synthetic aperture radar (SAR) images show that compared to normal GP, GP algorithm proposed here finds effective composite operators more quickly.

Fingerprint Classification Based on Learned Features

We present a fingerprint classification approach based on a novel feature-learning algorithm. Unlike current research for fingerprint classification that generally uses well defined meaningful features, our approach is based on Genetic Programming (GP), which learns to discover composite operators and features that are evolved from combinations of primitive image processing operations. Our experimental results show that our approach can find good composite operators to effectively extract useful features.

Integrating Relevance Feedback Techniques for Image Retrieval

We introduce an image relevance reinforcement learning (IRRL) model for integrating existing RF techniques in a content-based image retrieval system. Various integration schemes are presented and a long-term shared memory is used to exploit the retrieval experience from multiple users. The experimental results manifest that the integration of multiple RF approaches gives better retrieval performance than using one RF technique alone. Further, the storage demand is significantly reduced by the concept digesting technique.

On Labeling Noise and Outliers for Robust Concept Learning for Image Databases

Recently mixture model has been used to model image databases. The retrieval experiences derived from multiple users' relevance feedbacks have been used to improve model fitting in a semi-supervised manner. However, the mixture model for image databases remains as a challenging task since the database may contain clutter and outliers, and labelling information derived from multiple users may be inconsistent. Thus, neither the mixture model nor the labelling information is as ideal as most of the researchers have previously assumed. We (a) address the problems of the noise disturbances for both mixture model and users' labelling information, (b) propose to process retrieval experiences in an intelligent manner using Bayesian analysis, (c) present a robust mixture model fitting algorithm to achieve visual concept learning, and (d) construct a concept-based indexing structure for efficient search of the database. The experimental results on a Corel image set show the correctness of our retrieval experience analysis, the effectiveness of the proposed concept learning approach, and the improvement of retrieval performance based on the indexing structure.

Synthesizing Feature Agents Using Evolutionary Computation

Genetic programming (GP) with smart crossover and smart mutation is proposed in our research to discover integrated feature agents that are evolved from combinations of primitive image processing operations to extract regions-of-interest (ROIs) in remotely sensed images. Smart crossover and smart mutation identify and keep the effective components of integrated operators called "agents" and significantly improve the efficiency of GP. Our experimental results show that compared to normal GP, our GP algorithm with smart crossover and smart mutation can find good agents more quickly during training to effectively extract the regions-of-interest and the learned agents can be applied to extract ROIs in other similar images.

Functional Template-based SAR Image Segmentation

We present an approach to automatic image segmentation, in which user selected sets of examples and counter-examples supply information about the specific segmentation problem. In our approach, image segmentation is guided by a genetic algorithm which learns the appropriate subset and spatial combination of a collection of discriminating functions, associated with image features. The genetic algorithm encodes discriminating functions into a functional template representation, which can be applied to the input image to produce a candidate segmentation.

Learning Composite Operators for Object Detection

We have learned through this research to discover composite operators and features that are evolved from combinations of primitive image processing operations to extract regions-of-interest (ROIs) in images. Our approach is based on genetic programming (GP). The motivation for using GP is that there are a great many ways of combining these primitive operations and the human expert, limited by experience, knowledge and time, can only try a very small number of conventional ways of combination. Genetic programming, on the other hand, attempts many unconventional ways of combination that may never be imagined by human experts. In some cases, these unconventional combinations yield exceptionally good results. Our experimental results show that GP can find good composite operators, that consist of primitive operators designed by us, to effectively extract the regions of interest in images and the learned composite operators can be applied to extract ROIs in other similar images.

Genetic Algorithm Based Feature Selection for Target Detection in SAR Images

A genetic algorithm (GA) approach is presented to select a set of features to discriminate the targets from the natural clutter false alarms in SAR images. A new fitness function based on minimum description length principle (MDLP) is proposed to drive GA and it is compared with three other fitness functions. Experimental results show that the new fitness function outperforms the other three fitness functions and the GA driven by it selected a good subset of features to discriminate the targets from clutters effectively.

Local Discriminative Learning for Pattern Recognition

Local discriminative learning methods approximate a target function directly by partitioning the feature space into a set of local regions, and appropriately modeling a simple input-output relationship (function) in each one. We present a new method for judiciously partitioning the input feature space in order to accurately represent the target function. The method accomplishes this by approximating not only the target function itself but also its derivatives. As such, the method partitions the input feature space along those dimensions for which the class probability function changes most rapidly, thus minimizing bias. The efficacy of the method is validated using a variety of simulated and real-world data.

Probabilistic Feature Relevance Learning for Content-Based Image Retrieval

Most of the current image retrieval systems use "one-shot" queries to a database to retrieve similar images. Typically a K-nearest neighbor kind of algorithm is used, where weights measuring feature importance along each input dimension remain fixed (or manually tweaked by the user), in the computation of a given similarity metric. In this paper, we present a novel probabilistic method that enables image retrieval procedures to automatically capture feature relevance based on user's feedback and that is highly adaptive to query locations. Experimental results are presented that demonstrate the efficacy of our technique using both simulated and real-world data.

Learning to Perceive Objects for Autonomous Recognition

Machine perception techniques that typically used segmentation followed by object recognition lacked the required robustness to cope with the large variety of situations encountered in real-world navigation. Many existing techniques were brittle in the sense that even minor changes in the expected task environment (e.g., different lighting conditions, geometrical distortion, etc.) could severely degrade the performance of the system or even make it fail completely. Presented is a system that achieved robust performance by using local reinforcement learning to induce a highly adaptive mapping from input images to segmentation strategies for successful recognition which was verified through experiments on a large set of real images of traffic signs.This was accomplished by using the confidence level of model matching as reinforcement to drive learning. Local reinforcement learning gave rise to better improvement in recognition performance.

Probabilistic Feature Relevance Learning for Content-Based Image Retrieval

Most of the image retrieval systems used “one-shot” queries to a database to retrieve a similar image and typically a K-nearest neighbor kind of algorithm was used, where weights measuring feature importance along each input dimension remain fixed (or manually tweaked by the user), in the computation of a given similarity metric. However, the similarity did not vary with equal strength or in the same proportion in all directions in the feature space emanating from the query image. The manual adjustment of these weights was time consuming and exhausting and required a very sophisticated user. A probabilistic method that enabled image retrieval procedures to automatically capture feature relevance based on user’s feedback and that was highly adaptive to query locations is presented. Our findings demonstrate the efficacy of our technique using both simulated and real-world data.

Closed-Loop Object Recognition Using Reinforcement Learning

Current computer vision systems are not robust enough for most real-world applications. In contrast, the system presented here achieves robust performance by using reinforcement learning to induce a mapping from input images to corresponding segmentation parameters. This is accomplished by using the confidence level of model matching as a reinforcement signal for a team of learning automata to search for segmentation parameters during training.

Delayed Reinforcement Learning for Adaptive Image Segmentation

Generally, object recognition systems are open loop with no feedback between levels and assuring their robustness is a key challenge in computer vision and pattern recognition research. A robust closed-loop system based on “delayed” reinforcement learning is introduced in this paper. The parameters of a multilevel system employed for model-based object recognition are learned. The approach systematically controls feedback in a multilevel vision system and shows promise in approaching a long-standing problem in the field of computer vision and pattern recognition.

Closed-Loop Object Recognition Using Reinforcement Learning

Computer vision systems whose basic methodology was open-loop or filter type typically used image segmentation followed by object recognition algorithms. However those systems were not robust for most real-world applications. In contrast, the system presented achieved robust performance by using reinforcement learning to induce a mapping from input images to corresponding segmentation parameters. This was accomplished by using the confidence level of model matching as a reinforcement signal for a team of learning automata to search for segmentation parameters during training. The use of the recognition algorithm as part of the evaluation function for image segmentation gave rise to significant improvement of the system performance by automatic generation of recognition strategies.

Delayed Reinforcement Learning for Adaptive Image Segmentation and Feature Extraction

Object recognition was a multilevel process requiring a sequence of algorithms at low, intermediate, and high levels. Generally, such systems were open loop with no feedback between levels and ensuring their robustness was a key challenge in computer vision and pattern recognition research. A robust closed-loop system based on “delayed” reinforcement learning was introduced and the parameters of a multilevel system employed for model-based object recognition were learned. The method improved recognition results over time by using the output at the highest level as feedback for the learning system. It was experimentally validated by learning the parameters of image segmentation and feature extraction and thereby recognizing 2-D objects. The approach systematically controlled feedback in a multi-level vision system and showed promise in approaching a long-standing problem in the field of computer vision and pattern recognition.

Adaptive Target Recognition Using Reinforcement Learning

Research focused on using reinforcement learning to improve the performance of a SAR recognition engine is presented. We developed a learning algorithm which can direct the SAR recognition engine to perform at or close to a particular user prespecified performance point on the ROC curve of the engine.

Local Reinforcement Learning for Object Recognition

Computer vision systems whose basic methodology was open-loop or filter type typically used image segmentation followed by object recognition algorithms. These systems were not robust for most real-world applications. In contrast, the system presented here achieved robust performance by using local reinforcement learning to induce a highly adaptive mapping from input images to segmentation strategies. This was accomplished by using the confidence level of model matching as reinforcement to drive learning. The system was verified through experiments on a large set of real images.

Oracle: An Integrated Learning Approach for Object Recognition

Model-based object recognition had become a popular paradigm in computer vision research and in most of the model-based vision systems, the object models used for recognition were generally a priori given (e.g. obtained using a CAD model). For many object recognition applications , it was not realistic to utilize a fixed object model database with static model features, but it was desirable to have a recognition system capable of performing automated object model acquisition and refinement. In order to achieve these capabilities, we developed a system called ORACLE (Object Recognition Accomplished through Consolidated Learning Expertise) that used two machine learning techniques known as Explanation-Based Learning (EBL) and Structured Conceptual Clustering (SCC) combined in a synergistic manner. As compared to systems which learned from numerous positive and negative examples, EBL allowed the generalization of object model descriptions from a single example and then constructed an efficient classification tree which was incrementally built and modified over time. Learning from experience was used to dynamically update the specific feature values of each object. These capabilities provided a dynamic object model database which allowed the system to exhibit improved performance over time.

Adaptive Object Detection Based on Modified Hebbian Learning

The focus of this study was the issue of developing self-adapting automatic object detection systems for improving their performance. Two general methodologies for performance improvement were first introduced. They were based on parameter optimizing and input adapting. Different modified Hebbian learning rules were developed to build adaptive feature extractors which transformed the input data into a desired form for a given algorithm. To show its feasibility, an input adaptor for object detection was designed as an example and tested using multisensor data (optical, SAR, and FLIR).

Closed-Loop Object Recognition Using Reinforcement Learning

Computer vision systems whose basic methodology was open-loop or filter type typically used image segmentation followed by object recognition algorithms, but these systems were not robust for most real world applications. In contrast, the system presented here achieved robust performance by using reinforcement learning to induce a mapping from input images to corresponding segmentation parameters. This was accomplished by using the confidence level of model matching as a reinforcement signal for a team of learning automata to search for segmentation parameters during training. The use of the recognition algorithm was part of the evaluation function for image segmentation and gave rise to significant improvement of the system performance by automatic generation of recognition strategies. The system was verified through experiments on sequences of color images with varying external conditions.

Delayed Reinforcement Learning for Closed-Loop Object Recognition

Object recognition was a multi-level process requiring a sequence of algorithms at low, intermediate and high levels. Generally, such systems were open loop with no feedback between levels and ensuring their robustness was a key challenge in computer vision research. A robust closed-loop system based on “delayed” reinforcement learning is introduced and the parameters of a multi-level system employed for model-based object recognition are learned. The method improved recognition results over time by using the output at the highest level as feedback for the learning system. It had been experimentally validated by learning the parameters of image segmentation and feature extraction and thereby recognized 2-D objects. The approach systematically controlled feedback in a multi-level vision system and provided a potential solution to a long-standing problem in the field of computer vision.

Adaptive Image Segmentation Using Genetic and Hybrid Search Methods

One of the fundamental weaknesses of computer vision systems used in practical applications was their inability to adapt the segmentation process as real-world changes occurred in the image. Presented is the first closed loop image segmentation system which incorporated a genetic algorithm to adapt the segmentation process to changes in image characteristics caused by variable environmental conditions such as time of day, time of year, clouds, etc. The segmentation problem was formulated as an optimization problem and the genetic algorithm efficiently searched the hyperspace of segmentation parameter combinations to determine the parameter set which maximized the segmentation quality criteria. The goals of the adaptive image segmentation system were to provide continuous adaptation to normal environmental variations, to exhibit learning capabilities, and to provide robust performance when interacting with a dynamic environment. Also presented are experimental results which demonstrated learning and the ability to adapt the segmentation performance in outdoor color imagery.

Adaptive Image Segmentation Using a Genetic Algorithm

One of the fundamental weaknesses of computer vision systems used in practical applications was their inability to adapt the segmentation process as real-world changes occurred in the image. Presented is the first closed loop image segmentation system which incorporated a genetic algorithm to adapt the segmentation process to changes in image characteristics caused by variable environmental conditions such as time of day, time of year, clouds, etc. The segmentation problem was formulated as an optimization problem and the genetic algorithm efficiently searched the hyperspace of segmentation parameter combinations to determine the parameter set which maximized the segmentation quality criteria. The goals of the adaptive image segmentation system were to provide continuous adaptation to normal environmental variations, to exhibit learning capabilities, and to provide robust performance when interacting with a dynamic environment. Also presented are experimental results which demonstrated learning and the ability to adapt the segmentation performance in outdoor color imagery.

Enhancing a Self-Organizing Map through Near-Miss Injection

Kohonen’s self-organizing map (SOM) was viewed from the aspect of distinguishing different classes of feature vectors. Although a well trained SOM could convert the most important similarity relationships among the input feature vectors of the same class into the spatial relationships among the responding neurons, it lacked the power to exclude the near-miss feature vectors that belonged to another class. In order to use SOM as a classifier, we developed an algorithm called near-miss injection which, when used in conjunction with Kohonen’s SOM algorithm, could build a more “faithful” map for a given class that covered less feature vectors from another class. A 2-class classifier was built upon the trained SOM, and experimental results are shown on synthetic data.

Learning-Based Control of Perception for Mobility

Machine perception played an important role in any intelligent system, and in particular, guiding an autonomous mobile agent. Machine perception techniques had progressed significantly, however perception systems were still plagued by a lack of flexibility and an inadequacy in performance speed for use in real-time tasks. To overcome these problems, we applied integrated learning techniques to a perception system that was based on a selective sensing paradigm. The incorporation of multiple learning algorithms at different levels in our perception system provided a great deal of flexibility and robustness when performing different perceptual tasks. Making use of a selective sensing paradigm allowed the system to eliminate a large amount of non-pertinent sensory data so that processing speed was greatly increased. We implemented such a perception system to be used on an autonomous mobile agent.