Tensorflow Object Detection API in Flutter

This article explains you how to use tensorflow object detection api in flutter application to recognize object. We already implemented bar code/qr code scanner in flutter application.

In this flutter example, we are using Tflite flutter package to detect objects in flutter app. TensorFlow Lite framework provides two files model.tflite and labels.txt which we need to add in our source code. model.tflite is trained model and lables.txt contains all required labels.

 

Add dependencies in pubspec.yaml file

To detect object, we need to use camera preview. So we are using camera flutter package to recognize objects. tflite package is used for object detection. We need to add this two required libraries in our pubspec.yaml file.

camera: ^0.9.2
tflite: ^1.1.2
 
 

Add required Assets in pubspec.yaml file

We need to add tflite and txt files in root directory of our source code and register in pubspec.yaml file

assets:
    – assets/ssd_mobilenet.tflite
    – assets/ssd_mobilenet.txt
 
 

Get Available Cameras for Tensorflow Object Detection

First we need to get all available cameras using camera flutter package and add in array.

List<CameraDescription> listCameras;

Future<void> main() async {
WidgetsFlutterBinding.ensureInitialized();
listCameras = await availableCameras();
runApp(MyApp());
}

 
 

Load Assets in Flutter App

We need to load our assets files using loadModel method of Tflite class.

@override
void initState() {
super.initState();
loadAssets();
}

Future loadAssets() async {
Tflite.close();
await Tflite.loadModel(
model: “assets/ssd_mobilenet.tflite”,
labels: “assets/ssd_mobilenet.txt”);
}

 
 

Initialize Camera Controller

CameraController myCameraController;
CameraImage myCameraImage;

@override
void initState() {
super.initState();
initCamera();
}

initCamera() {
myCameraController = CameraController(widget.listCameras[0], ResolutionPreset.medium);
myCameraController.initialize().then((value) {
setState(() {
myCameraController.startImageStream((image) => {
myCameraImage = image,
detectObject(),
});
});
});
}

 

@override
void dispose() {
super.dispose();
myCameraController.stopImageStream();
Tflite.close();
}

 
 

Tensorflow Object Detection

detectObject() async {
listRecognitions = await Tflite.detectObjectOnFrame(
bytesList: myCameraImage.planes.map((plane) {
return plane.bytes;
}).toList(),
imageWidth: myCameraImage.width,
imageHeight: myCameraImage.height,
imageMean: 127.5,
imageStd: 127.5,
threshold: 0.4,
numResultsPerClass: 1,
);

setState(() {
myCameraImage;
});
}

 
 

Highlight Detected Objects

After detecting object in flutter app, we need to highlight it with rectangular shape.

List<Widget> highlightRecognizedObjects(Size screen) {
if (listRecognitions == null) return [];

double factorX = screen.width;
double factorY = screen.height;

Color highlightObjectColor = Colors.orangeAccent;

return listRecognitions.map((result) {
return Positioned(
left: result[“rect”][“x”] * factorX,
top: result[“rect”][“y”] * factorY,
width: result[“rect”][“w”] * factorX,
height: result[“rect”][“h”] * factorY,
child: Container(
decoration: BoxDecoration(
borderRadius: BorderRadius.all(Radius.circular(12)),
border: Border.all(color: Colors.orangeAccent, width: 3),
),
child: Text(
“${result[‘detectedClass’]} ${(result[‘confidenceInClass’] * 100).toStringAsFixed(0)}%”,
style: TextStyle(
background: Paint()..color = highlightObjectColor,
color: Colors.black,
fontSize: 18,
),
),
),
);
}).toList();
}

 
 

Display Camera Preview for Tensorflow Object Detection

@override
Widget build(BuildContext context) {
Size size = MediaQuery.of(context).size;
List<Widget> listDetectedObjects = [];

listDetectedObjects.add(
Positioned(
top: 0.0,
left: 0.0,
width: size.width,
height: size.height – 100,
child: Container(
height: size.height – 100,
child: (!myCameraController.value.isInitialized)
? Container()
: AspectRatio(
aspectRatio: myCameraController.value.aspectRatio,
child: CameraPreview(myCameraController),
),
),
),
);

if (myCameraImage != null) {
listDetectedObjects.addAll(highlightRecognizedObjects(size));
}

return Scaffold(
backgroundColor: Colors.black,
body: Container(
margin: EdgeInsets.only(top: 50),
color: Colors.black,
child: Stack(
children: listDetectedObjects,
),
),
);
}

 
 
tensorflow object detection api
 

One thought on “Tensorflow Object Detection API in Flutter

Leave a Reply