🚨 We are hiring! 🚨
The ACL Rolling Review is looking for motivated individuals to join our team! If you're passionate about NLP and want to contribute to the community, apply now!
Editorial Assistant -
Workflow Manager -
Regarding ACL, some graphs in case they’re helpful: cumulative score distributions for meta reviews (left) and paper avg reviews (right) for the Nov. deadline. Caveat: as with all conferences, decisions are not determined by scores alone, but rather a holistic view of the paper.
We have a new tutorial for reviewers from
@annargrs
and
@IAugenstein
! This covers all aspects of writing a review, from preparation before reviewing, to basic principles of reading for review and providing useful comments, as well as common pitfalls.
ARR is now accepting submissions! Please see for an overview of the submission form and link to the submission site. Submit by 5/15 to be eligible for
@emnlpmeeting
!
#NLProc
We received 2900+ November submissions. To ensure that conflicts of interest are handled appropriately and review assignments made with reviewers' expertise in hand, please complete your profile *today*. A guide to how to do this can be found at
To all authors, reviewers, and AEs: make sure to *complete* your OpenReview profile by:
1. adding a link to your Semantic Scholar page
2. adding a link to your DBLP page and clicking "import papers from DBLP"
This is absolutely *necessary* for good paper-reviewer/AE matching.
ARR needs your help! We received 5800+ submissions for the June cycle, but with our current capacity, we can handle only half of these submissions. We can't start the review process without significant help. (1/2)
We wanted to give a few background details on what's happening at ARR, so we wrote a short status report. Read it to find statistics, challenges, and new initiatives.
ARR June/EMNLP 2024 review deadline was July 19th. Thank you to the reviewers who met the deadline. We still await many reviews. Please submit ASAP, as the authors rely on your feedback by July 25th.
We have a draft CFP . Submission deadline will be the 15th of each month, starting 5/15/21. Authors who want their ARR submissions to be considered for EMNLP must submit by 5/15/21.
Thanks to
@mjpost
@HinrichSchuetze
Ada Wan for CFP corrections!
ARR's first deadline has passed, and many of the submissions are available as anonymous preprints. We're working on an account that you can follow separately to see each submission, but for now, see this page:
Thanks to everyone for your hard work in the reviewing process! Public Service Announcement: once you sign up as a reviewer, this carries over each month. However, you can mark yourself unavailable for some period, or request a max load, using the following forms:
We are happy to announce the ARR mentorship program. Are you a junior NLP researcher who doesn’t yet feel ready to review for ARR? Consider becoming a mentee. Already reviewing for ARR? Consider becoming a mentor! More information at .
The 2024 *CL conferences (EACL, NAACL, & ACL) are all using ARR as the only submission and review system. Read here for more details about this coordinated effort:
@eaclmeeting
@naacl
@aclmeeting
Effective the 15 April 2022 deadline, with permission of the ACL executive and the ACL committee on reviewing, ARR will be moving to a six-week cycle with deadlines every six weeks. Further details:
Hi Everyone! First, thanks to all reviewers and AEs who volunteered to help. We've sent review assignments for November, and gotten a lot of (justified) feedback that the review load is high and the time is short. Why is this? and what are we planning to do about it? :thread:
@ReviewAcl
is implementing an important change in the submission process for upcoming cycles. Each submission will now need to provide information regarding reviewing commitment. Exceptions apply. Read more here:
Good news: the timeline has shifted so reviewers have more time for reviews. There will be emails sent out, but the new deadline for reviewing is ***Dec 27th.***
@ReviewAcl
Breaking news: The
@aclmeeting
program chairs have pushed out the ACL 2022 commitment date to Jan 15, thus allowing for a reviewing period of 11/29-12/27. Updated timeline at . Emails will go out to AEs and reviewers.
To all authors, reviewers, and AEs: make sure to *complete* your OpenReview profile by:
1. adding a link to your Semantic Scholar page
2. adding a link to your DBLP page and clicking "import papers from DBLP"
This is absolutely *necessary* for good paper-reviewer/AE matching.
For the current ARR cycle we switched to including the responsible NLP checklist inside the submission form rather than as a pdf. That switch did not go smoothly and some authors temporarily saw a pdf upload option. We have now resolved the issue, so all authors see the new form.
If you haven't taken a look recently, check out our stats page! There are some new views on linguistic / geographic diversity, and we also have stats from January. Many thanks to
@SebastinSanty
for his work on this!
We had some technical issues with the ARR reviewer volunteer form being full, very sorry for everyone who tried to volunteer but couldn't! We have a new, fixed form now, so please sign up if you can help out reviewing:
Reviews and meta-reviews for October submissions are now available. Not every paper is complete yet (thanks for your patience!), but everything finished is visible. ~Stephen
If you, like Victor, can imagine a world without Softconf, consider submitting to ACL Rolling Review, which uses
@openreviewnet
. Next deadline June 15!
Note: you cannot commit your paper+reviews to NAACL and then submit a further revision to ARR.
In general, as long as an ARR-reviewed paper is under consideration for a venue, it cannot be resubmitted to ARR.
We are sending invitations out to action editors and reviewers. And we have *submissions*!! Thank you for volunteering, responding to invites, submitting! Please create / update your
@openreviewnet
profile so we can do good COI handling and paper matching.
#NLProc
We'll have a report out soon with more details, but two reason reviews return late to authors is 1) a small reviewer pool, and 2) late reviews.
Especially as deadline season approaches, it will be critical to commit to review, and to review in time.
We're pleased to announce that Malihe Alikhani has accepted to take on the role of Ethics co-chair for ARR. Malihe works towards designing inclusive and equitable language technologies. She develops systems that can communicate and collaborate with diverse populations,
Update with correct numbers: We have received a healthy number of submissions for the October 2023 ARR cycle. Currently, we have 1262 active submissions, which are being reviewed by a team of 1,437 reviewers and 266 Area Chairs.
In preparation for a lot of reviewing, the ACL and ARR program chairs have made two tutorials for common tasks on OpenReview:
Reviewers:
Action Editors:
These are also linked in the relevant pages on the website.
We’re delighted to share the news that Jun Suzuki has joined the ACL RR team as an Editor-in-Cheif. Jun is a full professor at Tohoku University, working extensively on AI/ML/NLP.
@ReviewAcl
It looks like November-cycle reviews are out but metareviews aren't. Known issue? (I see none for my any of my four submissions, so this probably isn't just the few percent of papers that don't have them at all.)
Heads up to authors: if you want to submit to arxiv and remain outside the anonymity period for the Nov deadline, today (AoE) is your last day. The deadline is based on *submission* time, not *publication* time.
New ACL Rolling Review Data Collection (ARR-DC) protocol launched during April cycle. To read more about this visit this link:
Consider donating your peer-review data! You can always revisit this decision as needed.
Reviews and meta-reviews for October submissions are now available. Not every paper is complete yet (thanks for your patience!), but everything finished is visible. ~Stephen
If you want to serve as a volunteer to review for ARR, please write to us at support
@aclrollingreview
.org and provide the following: your name, link to your DBLP profile, and your OpenReview ID.
We require 5+ papers (in NLP venues) in DBLP and at least one in the last 5 years.
ARR is looking for volunteer to serve as a Co-CTO and improve the tech infrastructure.
Preferred qualifications:
a) min 5 years experience with NLP research
b) skilled with git, command line tools, python, and simple html.
Go check out
@ARRPreprints
, where we just posted many of the anonymous preprints from the November deadline! not all papers are posted: I ran into a twitter posting limit while running the bot. I'll try again tomorrow. ~Stephen
Just a note to say that we've seen this feedback, thank you. The Editors work with conf PCs to collaboratively write this form, and it is updated at regular intervals, taking into account current reviewing cycles (it's potentially confusing to change the form mid-review cycle).
Dear ARR authors. We have released reviews for all papers with completed reviews and meta-reviews for the May iteration. A significant number of papers still have reviews in progress, and we will be releasing reviews for these papers ASAP.
We're thrilled to announce that Vincent Ng has joined the ARR EiC team! He is a Professor at the University of Texas at Dallas. For the past 20 years or so, he has been working on problems in the areas of discourse, argumentation, and source code analysis, ..
ARR 2024 June authors: Make sure the reviewing volunteers you specify in the submission form are qualified reviewers (i.e., people with at least 3 papers in the ACL Anthology). See for details.
ARR and
@aclmeeting
jointly guarantee that all papers submitted for the November ARR deadline (as well as those submitted in previous months, Sep and Oct) will be fully processed in time to be committed for ACL 22.
Reminder that the regular Rolling Review deadline (5/15) is coming up in just over a week! Submit this month so reviews will be ready in time for
@emnlpmeeting
ARR is now accepting submissions! Please see for an overview of the submission form and link to the submission site. Submit by 5/15 to be eligible for
@emnlpmeeting
!
#NLProc
FYI: there will be a slightly modified timeline for the November cycle. See details here: Notably: reviews will be back before the Jan 7 ACL deadline. ~Stephen
If you need a letter of recognition for your service as a reviewer/action editor for ARR, please write to us at support_at_aclrollingreview_dot_org. Please mention your Full name, OpenReview ID, and the cycle for which you are requesting the letter.
Join us to learn more about the ACL Rolling Review (ARR) initiative. A lot has changed since ARR got started; come and hear all about it during the info session @ EACL. We will also have time for Q&A and/or feedback from participants.
Thank you to all of the authors who submitted to the May iteration of ARR, 87% of papers have received all reviews and we expect to send the remainder out in the next few days. Many apologies to those who got reviews late. 1/2
Reviews for all ARR September (and August) submissions that we have obtained until this point have now been released and should be visible to the authors in OpenReview. The processing of some submissions is still incomplete (e.g., missing a review or missing the meta-review).
If you are invited to review/AC but haven't responded, please respond positively. If you have responded, please consider raising your load by as much as you can handle. (2/2)
We have openings for our "communications team", especially helping to improve communications turnaround time (especially email). Please consider applying!
You may have read about the CVPR social media ban and wondered about ARR. There is no explicit ban, but our CfP says:
“…we ask authors not to advertise on social media or take other actions that would further compromise two-way anonymized reviewing during the anonymity period.”
We have added a dates page:
Authors, refer to this page for a list of ARR participating venues and dates.
Conference and workshop organizers, refer to this page to figure out what the last ARR submission date should be for submissions to your venue.
The Nov deadline has only just passed, but we are already thinking about the Dec deadline. There are a few changes to the process -- you can read about them here.
We are preparing for the February cycle of ARR, the last before ACL commitment, and need all reviewers and AEs to complete their availability forms (and other forms too if you haven't done them already!):
Reviewers:
AEs:
Remember that ARR is the sole reviewing system for ACL and NAACL. You won't be getting separate review requests from those conferences! We'd encourage you to treat ARR review requests the same as you would those from ACL or NAACL.
If you have received your reviews before December 15, 2021, and are planning to commit to ACL 2022, please remember *NOT* to post a deanonymized preprint (e.g., in arXiv) on or after December 15, 2021.
To clarify: ARR follows the ACL review policies. A paper can be posted online deanonymized unless:
1. It is currently under active review at ARR, or will be within one month.
2. It is currently submitted for consideration to an ACL venue, or will be within one month
I spoke with the reviewing committee and got an important clarification. You can post your paper non-anonymously after ARR reviews and before submission to a conference, but not within the anonymity window of that conference. This resolves the issue for me.
We've had several emails about this, and it's certainly confusing. These are usually emergency review requests, and they may go out after the deadline. We plan to change the email to say: take the time you need to do a good review, with the understanding that authors are waiting.
@ReviewAcl
- I got two papers to review today (19th oct) with a deadline of 15th Oct! Both of them. Even Gods can't do this :D Since there is no AC info, I had to contact the generic support. I reviewed for *ACL even the years did not submit. ARR is making it impossible for me.
@ReviewACL
is looking for a few extra members of the technical staff, who implement the OpenReview interface, automated systems for reviewer management, paper-reviewer matching, etc. Please contact cto
@aclrollingreview
.org with a quick self-introduction if interested!
FYI, in case it's helpful: Reviewers and Action Editors are not mutually anonymous in ARR. Reviewers can contact AEs either directly through email, or through OpenReview (see the FAQ).
We sharing a full report of the process very soon, but the main issues causing late reviews were the learning curve of switching to new reviewing software, and establishing reviewer management processes. We hope things will be much more organized in coming iterations! 2/2
Re: ICLR withdrawal. If your paper was:
* on arXiv before the required 1 month anonymity period, and withdrawn from ICLR within that time period, then you can submit to ARR
* NOT on arXiv but was deanonymized b/c of withdrawal from ICLR, you have to wait a month to submit to ARR
If you are not yet in the ARR reviewer/AC pool and would be interested in serving as an area chair for ARR 2024 June, please fill out this form by June 20:
To clarify: ARR does not require that identities of action editors and reviewers are hidden from each other. At the moment, this happens almost de facto because of the way welcome emails and website interface are set up. We'll change it for the October deadline!
@tpimentelms
@ReviewAcl
I don't understand ARR's intentions regarding action editor anonymity. Why not identify them in the invitation email? It might actually improve acceptance rates if the potential reviewer can see who picked them for the paper.
For June papers that have reviewer volunteer issues, we’ve created a new form due to the limits hit by the previous one. Please respond here to provide volunteers: . The form about other issues (e.g. availability) remains the same:
Starting with the July ARR cycle (submission deadline July 15), reviewers and AEs can declare their max loads (and unavailability, as the max load of 0) directly in OpenReview. More details:
Decisions on whether you can commit an incompletely processed ARR submission to a venue are up to the venue organizers. ACL 22 PCs (working closely with ARR) decided that only fully processed submissions (i.e., with three reviews and the meta-review) can be committed to ACL 22.