{"id":36,"date":"2019-04-02T15:11:56","date_gmt":"2019-04-02T14:11:56","guid":{"rendered":"https:\/\/ase.in.tum.de\/cseet2020\/?page_id=36"},"modified":"2019-07-09T15:33:48","modified_gmt":"2019-07-09T14:33:48","slug":"review-process","status":"publish","type":"page","link":"https:\/\/ase.in.tum.de\/cseet2020\/index.php\/review-process\/","title":{"rendered":"Double-Blind Review Process for Research Track Papers"},"content":{"rendered":"\n
CSEE&T 2020 will employ a full double-blind review process for research track papers<\/a>. The papers submitted must not reveal the authors\u2019 identities. Identities will only be revealed to the reviewers when final decisions have been made. The Organizers <\/a>reserve the right to desk-reject without review any submission that does not adhere to the instructions outlined below.<\/p>\n\n\n\n This document draws on other guides from ASE 2017, the CAV 2016 FAQ<\/a>, David Walker\u2019s double-blind reviewing FAQ from POPL 2015, and includes content from the ICSE 2018<\/a> and ICSA 2019<\/a> site. Some sections have been copied verbatim.<\/em><\/p>\n\n\n\n Authors must make every effort to honor the double-blind review process. In case of questions, please contact the PC Chairs<\/a>. Please adhere to the following checklist:<\/p>\n\n\n\n Q<\/strong>: What exactly do I have to do to anonymize my paper?<\/em><\/p>\n\n\n\n A<\/strong>: Your job is not to make your identity undiscoverable but simply to make it possible for our reviewers to evaluate your submission without having to know who you are. The specific guidelines are simple: omit authors\u2019 names from your title page (or list them as \u201comitted for submission\u201d), and when you cite your own work, refer to it in the third person.<\/p>\n\n\n\n Q<\/strong>: I would like to provide supplementary material for consideration, e.g., the code of my implementation or proofs of theorems. How do I do this?<\/em><\/p>\n\n\n\n A<\/strong>: On the submission site there will be an option to submit supplementary material along with your main paper. This supplementary material need not<\/strong> be anonymized; it will only be revealed to reviewers after they have submitted their review of your paper and learned your identity. Reviewers are under no obligation to look at this material. The submission itself is the object of review and so it should strive to convince the reader of at least the plausibility of reported results; supplemental material only serves to confirm, in more detail, the idea argued in the paper. Of course, reviewers are free to change their review upon viewing supplemental material (or for any other reason). For those authors who wish to supplement, we encourage them to mention the supplement in the body of the paper so reviewers know to look for it, if necessary. E.g., \u201cThe proof of Lemma 1 is included in the non-anonymous supplemental material submitted with this paper.\u201d<\/p>\n\n\n\n Q<\/strong>: Is there a way for me to submit anonymous supplemental material which could be considered by a reviewer before she submits her review (rather than potentially non-anonymous material that can only be viewed afterward) ?<\/em><\/p>\n\n\n\n A<\/strong>: There is no official channel for doing this: the submission site only accepts potentially non-anonymous material. That said, authors have been known to release a TR, code, etc. via an anonymous hosting service, and to include a URL to that material in the paper. However, we discourage authors from using such tactics except for materials that cannot, for some reason, be uploaded to the official site (e.g., a live demo). We emphasize that authors should strive to make their paper as convincing as possible within the submission page limit, in case reviewers choose not to access supplemental material. Also, see the next question.<\/p>\n\n\n\n Q<\/strong>: Can I supplement my submission using a URL that links to auxiliary materials instead of submitting such materials to the EasyChair system directly?<\/em><\/p>\n\n\n\n A <\/strong>In general, we discourage authors from providing supplementary materials via links to external web sites. It is possible to change the linked items after the submission deadline has passed, and, to be fair to all authors, we would like to be sure reviewers evaluate materials that have been completed prior to the submission deadline. Having said that, it is appropriate to link to items, such as an online demo, that can\u2019t easily be submitted. Needless to say, attempting to discover the reviewers for your paper by tracking visitors to such a demo site would be a breach of academic integrity. Supplementary items such as PDFs should always be uploaded to EasyChair.<\/p>\n\n\n\n Q<\/strong>: I am building on my own past work on the WizWoz system. Do I need to rename this system in my paper for purposes of anonymity, so as to remove the implied connection between my authorship of past work on this system and my present submission?<\/em><\/p>\n\n\n\n A<\/strong>: No. The relationship between systems and authors changes over time, so there will be at least some doubt about authorship. Increasing this doubt by changing the system name would help with anonymity, but it would compromise the research process. In particular, changing the name requires explaining a lot about the system again because you can\u2019t just refer to the existing papers, which use the proper name. Not citing these papers runs the risk of the reviewers who know about the existing system thinking you are replicating earlier work. It is also confusing for the reviewers to read about the paper under Name X and then have the name be changed to Name Y. Will all the reviewers go and re-read the final version with the correct name? If not, they have the wrong name in their heads, which could be harmful in the long run.<\/p>\n\n\n\n Q<\/strong>: I am submitting a paper that extends my own work that previously appeared at a workshop. Should I anonymize any reference to that prior work?<\/em><\/p>\n\n\n\n A<\/strong>: No. But we recommend you do not use the same title for your new submission, so that it is clearly distinguished from the prior paper. In general there is rarely a good reason to anonymize a citation. One possibility is for work that is tightly related to the present submission and is also under review. But such works may often be non-anonymous. When in doubt, contact the Program Chairs<\/a>.<\/p>\n\n\n\n Q<\/strong>: Am I allowed to post my (non-blinded) paper on my web page? Can I advertise the unblinded version of my paper on mailing lists or send it to colleagues? May I give a talk about my work while it is under review?<\/em><\/p>\n\n\n\n A<\/strong>: As far as the authors\u2019 publicity actions are concerned, a paper under double-blind review is largely the same as a paper under regular (single-blind) review. Double-blind reviewing should not hinder the usual communication of results.<\/p>\n\n\n\n Q<\/strong>: Will the fact that this conference is double-blind have an impact on handling conflicts-of interest? <\/em><\/p>\n\n\n\n A<\/strong>: Authors will enter their conflicts of interest with the PC in the conference software system when they register their papers. This information will be used to avoid conflicts of interest in reviewing.<\/p>\n\n\n\n A conflict of interest is defined as a situation in which a reviewer can be viewed as being able to benefit personally in the process of reviewing a paper. For example, if areviewer is considering a paper written by a member of their own group, a current student, their advisor, their close collaborators (including recent coauthors) then the outcome of the review process can have direct benefit to the reviewer\u2019s own status.<\/p>\n\n\n\n As an author, you should list PC members which you believe have a conflict with you. Tihs conference will apply the ACM SIGSOFT conflict guidelines<\/a>. Also, please identify institutions with which you are affiliated; all employees or affiliates of these institutions will also be considered conflicted.<\/p>\n\n\n\n If a possible reviewer does not meet the above criteria, please\u00a0do not<\/em> identify him\/her as conflicted. Doing so could be viewed as an attempt to prevent a qualified, but possibly skeptical reviewer from reviewing your paper. If you nevertheless believe that a reviewer who does not meet the above criteria is conflicted, you may identify the person and send a note to the Program Chairs<\/a>.<\/p>\n\n\n\n Q<\/strong>: How should I refer to my own papers that are under consideration somewhere else?<\/em><\/p>\n\n\n\n A<\/strong>: If the work being cited is in press, it may be cited as \u201cAnonymous.\u201d We would remind authors that submissions to this conference must be original and not under consideration elsewhere.<\/p>\n\n\n\n Q<\/strong>: What should I do if I if I learn the authors’ identity? <\/em><\/p>\n\n\n\n A<\/strong>: If at any point you feel that the authors\u2019 actions are largely aimed at ensuring that potential reviewers know their identity, you should contact the Program Chairs<\/a>. Otherwise you should not treat double-blind reviewing differently from regular blind reviewing. In particular, you should refrain from seeking out information on the authors\u2019 identity, but if you discover it accidentally this will not automatically disqualify you as a reviewer. Use your best judgment.<\/p>\n\n\n\n Q<\/strong>: The authors have provided a URL to supplemental material. I would like to see the material but I worry they will snoop my IP address and learn my identity. What should I do?<\/em><\/p>\n\n\n\n A<\/strong>:You can use VPN, the Tor<\/a> browser, or proxies such as HideMe<\/a>, to anonymously access URLs for supplemental material.<\/p>\n\n\n\n Q<\/strong>: How do we handle potential conflicts of interest since I cannot see the author names?<\/em><\/p>\n\n\n\n A<\/strong>: The conference review system will ask that you identify conflicts of interest when you get an account on the submission system. Please see the Author FAQ above to decide how to identify conflicts. Feel free to also identify additional authors whose papers you feel you could not review fairly for reasons other than those given (e.g., strong personal friendship). The PC Chairs will be responsible for conflict management but can only work with the information provided. (Taken verbatim from <\/em>ICSE2018 webpage<\/em><\/a>)<\/em><\/p>\n\n\n\n For those who are interested in motivations for double blind reviewing, a very well argued, referenced and evidenced article in favour of double blind review processes for Software Engineering conferences can be found in the blog post by Claire Le Goues: https:\/\/www.cs.cmu.edu\/~clegoues\/double-blind.html<\/a>. <\/p>\n\n\n\n And here is a list of double-blind resources from Robert Feldt: http:\/\/www.robertfeldt.net\/advice\/double_blind_reviewing\/<\/a>.<\/p>\n\n\n\n And a more formal study of the subject by Moritz Beller\u200b\u200b and Alberto Bacchelli\u200b\u200b: https:\/\/peerj.com\/preprints\/1757\/<\/a> <\/p>\n","protected":false},"excerpt":{"rendered":" CSEE&T 2020 will employ a full double-blind review process for research track papers. The papers submitted must not reveal the authors\u2019 identities. Identities will only be revealed to the reviewers when final decisions have been made. The Organizers reserve the … Continue reading Author Checklist<\/strong><\/h4>\n\n\n\n
Frequently Asked Questions<\/strong><\/h4>\n\n\n\n
For reviewers<\/strong><\/h4>\n\n\n\n
<\/p>\n\n\n\nMore information about bias in merit reviewing<\/strong><\/h4>\n\n\n\n