It is well known that query-based attacks tend to have relatively higher successrates in adversarial black-box attacks. While research on black-box attacks is activelybeing conducted, relatively few studies have focused on pixel attacks thattarget only a limited number of pixels. In image classification, query-based pixelattacks often rely on patches, which heavily depend on randomness and neglectthe fact that scattered pixels are more suitable for adversarial attacks. Moreover, tothe best of our knowledge, query-based pixel attacks have not been explored in thefield of object detection. To address these issues, we propose a novel pixel-basedblack-box attack called Remember and Forget Pixel Attack using ReinforcementLearning(RFPAR), consisting of two main components: the Remember and Forgetprocesses. RFPAR mitigates randomness and avoids patch dependency byleveraging rewards generated through a one-step RL algorithm to perturb pixels.RFPAR effectively creates perturbed images that minimize the confidence scoreswhile adhering to limited pixel constraints. Furthermore, we advance our proposedattack beyond image classification to object detection, where RFPAR reducesthe confidence scores of detected objects to avoid detection. Experimentson the ImageNet-1K dataset for classification show that RFPAR outperformedstate-of-the-art query-based pixel attacks. For object detection, using the MSCOCOdataset with YOLOv8 and DDQ, RFPAR demonstrates comparable mAPreduction to state-of-the-art query-based attack while requiring fewer query. Furtherexperiments on the Argoverse dataset using YOLOv8 confirm that RFPAReffectively removed objects on a larger scale dataset. Our code is available athttps://github.com/KAU-QuantumAILab/RFPAR.