Tensorflow Object Detection API in Flutter
This article explains you how to use tensorflow object detection api in flutter application to recognize object. We already implemented bar code/qr code scanner in flutter application.
In this flutter example, we are using Tflite flutter package to detect objects in flutter app. TensorFlow Lite framework provides two files model.tflite and labels.txt which we need to add in our source code. model.tflite is trained model and lables.txt contains all required labels.
Add dependencies in pubspec.yaml file
To detect object, we need to use camera preview. So we are using camera flutter package to recognize objects. tflite package is used for object detection. We need to add this two required libraries in our pubspec.yaml file.
tflite: ^1.1.2
Add required Assets in pubspec.yaml file
We need to add tflite and txt files in root directory of our source code and register in pubspec.yaml file
– assets/ssd_mobilenet.tflite
– assets/ssd_mobilenet.txt
Get Available Cameras for Tensorflow Object Detection
First we need to get all available cameras using camera flutter package and add in array.
Future<void> main() async {
WidgetsFlutterBinding.ensureInitialized();
listCameras = await availableCameras();
runApp(MyApp());
}
Load Assets in Flutter App
We need to load our assets files using loadModel method of Tflite class.
void initState() {
super.initState();
loadAssets();
}
Future loadAssets() async {
Tflite.close();
await Tflite.loadModel(
model: “assets/ssd_mobilenet.tflite”,
labels: “assets/ssd_mobilenet.txt”);
}
Initialize Camera Controller
CameraImage myCameraImage;
@override
void initState() {
super.initState();
initCamera();
}
initCamera() {
myCameraController = CameraController(widget.listCameras[0], ResolutionPreset.medium);
myCameraController.initialize().then((value) {
setState(() {
myCameraController.startImageStream((image) => {
myCameraImage = image,
detectObject(),
});
});
});
}
@override
void dispose() {
super.dispose();
myCameraController.stopImageStream();
Tflite.close();
}
Tensorflow Object Detection
listRecognitions = await Tflite.detectObjectOnFrame(
bytesList: myCameraImage.planes.map((plane) {
return plane.bytes;
}).toList(),
imageWidth: myCameraImage.width,
imageHeight: myCameraImage.height,
imageMean: 127.5,
imageStd: 127.5,
threshold: 0.4,
numResultsPerClass: 1,
);
setState(() {
myCameraImage;
});
}
Highlight Detected Objects
After detecting object in flutter app, we need to highlight it with rectangular shape.
if (listRecognitions == null) return [];
double factorX = screen.width;
double factorY = screen.height;
Color highlightObjectColor = Colors.orangeAccent;
return listRecognitions.map((result) {
return Positioned(
left: result[“rect”][“x”] * factorX,
top: result[“rect”][“y”] * factorY,
width: result[“rect”][“w”] * factorX,
height: result[“rect”][“h”] * factorY,
child: Container(
decoration: BoxDecoration(
borderRadius: BorderRadius.all(Radius.circular(12)),
border: Border.all(color: Colors.orangeAccent, width: 3),
),
child: Text(
“${result[‘detectedClass’]} ${(result[‘confidenceInClass’] * 100).toStringAsFixed(0)}%”,
style: TextStyle(
background: Paint()..color = highlightObjectColor,
color: Colors.black,
fontSize: 18,
),
),
),
);
}).toList();
}
Display Camera Preview for Tensorflow Object Detection
Widget build(BuildContext context) {
Size size = MediaQuery.of(context).size;
List<Widget> listDetectedObjects = [];
listDetectedObjects.add(
Positioned(
top: 0.0,
left: 0.0,
width: size.width,
height: size.height – 100,
child: Container(
height: size.height – 100,
child: (!myCameraController.value.isInitialized)
? Container()
: AspectRatio(
aspectRatio: myCameraController.value.aspectRatio,
child: CameraPreview(myCameraController),
),
),
),
);
if (myCameraImage != null) {
listDetectedObjects.addAll(highlightRecognizedObjects(size));
}
return Scaffold(
backgroundColor: Colors.black,
body: Container(
margin: EdgeInsets.only(top: 50),
color: Colors.black,
child: Stack(
children: listDetectedObjects,
),
),
);
}
Pingback: Custom Radio Button in Flutter - CodingWithDhrumil